This manual describes the use and configuration of the 3G Bridge REST interface.

The REST interface is an alternative to the SOAP based interface, with similar functionalities. The only limitation is that local files cannot be specified as input files when submitting a job; only remote file specifications are possible. Aside from this, the REST interface is a superset of the SOAP interface.

In this manual, we assume that the project URL is http://example.com/myproject/, and the REST interface is deployed in download/wss/. That is, the base URL of the REST interface is http://example.com/myproject/download/wss/.

A requested entity or collection will respond with a table (list of associative arrays). This response will be rendered according to the format argument, which can be plain, html or json. The hdr argument enables (1) or disables (0) the rendering of a header if applicable. If the Content-Type of the request is 'application/json' , the response will be in JSON format. Otherwise, the default response format is plain.

• plain: The response will be rendered as plain text. Records are separated with '\n', attributes are separated with a single space. Separators can be overridden with the sep and rsep (record sepatator) arguments. A header is rendered if requested (hdr=1).
• html: The response is rendered as a HTML table. A header is rendered by default.
• json: The response is rendered as a JSON string.

Example:

http://example.com/myproject/download/wss/version

3G Bridge 1.9

http://example.com/myproject/download/wss/version?hdr=1

version
3G Bridge 1.9

http://example.com/myproject/download/wss/version?format=json

[{"version":"3G Bridge 1.9"}]

For plain format, the field separator can be overridden with the sep argument. The record separator can be overridden with rsep.

http://example.com/myproject/download/wss/jobs/093fd32b-4604-4c06-9ba9-d3aae7a6b80d?sep=|

093fd32b-4604-4c06-9ba9-d3aae7a6b80d|Cloud|cloud|CANCEL||--image=ami-00000021|||2012-09-11 11:20:06|

For most entities (not collections), specific attributes can be selected in the URL:

http://example.com/myproject/download/wss/jobs/093fd32b-4604-4c06-9ba9-d3aae7a6b80d/id+status+creation_time

93fd32b-4604-4c06-9ba9-d3aae7a6b80d CANCEL 2012-09-11 11:20:06

Most entities only support the GET verb. If not stated otherwise, the following output examples are the results of GET requests on the specified entities.

http://example.com/myproject/download/wss/version

3G Bridge 1.9

SOAP equivalent:

wsclient -m version -e http://example.com:8091/

http://example.com/myproject/download/wss/queues

Cloud audiveris 1
edgidemo audiveris 1 
edgidemo bwa-boinc 1
NULL dsp
MetaJob dsp
...

SOAP equivalent: —

http://example.com/myproject/download/wss/grids

Cloud
edgidemo
MetaJob
NULL

SOAP equivalent: —

1. All attributes:

   http://example.com/myproject/download/wss/grids/edgidemo

2. Only algorithm names:

   http://example.com/myproject/download/wss/grids/edgidemo/alg

edgidemo audiveris 1 
edgidemo bbgc 1 
edgidemo Blender 1 
edgidemo bwa-boinc 1
...

SOAP equivalent for 2nd case:

wsclient -m gridalgs -e http://example.com:8091/ -g edgidemo

http://example.com/myproject/download/wss/jobs?hdr=1

id grid alg status gridid args griddata tag creation_time metajobid
093fd32b-4604-4c06-9ba9-d3aae7a6b80d Cloud cloud CANCEL  --image=ami-00000021   2012-09-11 11:20:06 
0bba95f9-38ea-4cf9-8125-6bc3e93af154 Cloud cloud CANCEL  --image=ami-00000021   2012-09-13 10:40:07
...

SOAP equivalent: —

http://example.com/myproject/download/wss/jobs/093fd32b-4604-4c06-9ba9-d3aae7a6b80d

093fd32b-4604-4c06-9ba9-d3aae7a6b80d Cloud cloud CANCEL  --image=ami-00000021   2012-09-11 11:20:06

Any subset of the attributes can be selected. Consider the following SOAP commands:

wsclient -m status -e http://example.com:8091/ -j 093fd32b-4604-4c06-9ba9-d3aae7a6b80d

REST equivalent:

http://example.com/myproject/download/wss/jobs/093fd32b-4604-4c06-9ba9-d3aae7a6b80d/id+status

093fd32b-4604-4c06-9ba9-d3aae7a6b80d CANCEL

wsclient -m griddata -e http://example.com:8091/ -j 093fd32b-4604-4c06-9ba9-d3aae7a6b80d

REST equivalent:

http://example.com/myproject/download/wss/jobs/093fd32b-4604-4c06-9ba9-d3aae7a6b80d/id+griddata

093fd32b-4604-4c06-9ba9-d3aae7a6b80d d2048dd6-464e-47c9-a4d0-373a9de25bdb_e933a6f8-7365-4b58-846d-a38be7d4ba24_76a26b38-88fd-4ba3-912b-2b1e30ed74d6

http://example.com/myproject/download/wss/jobs/finished/id+status

093fd32b-4604-4c06-9ba9-d3aae7a6b80d FINISHED
0bba95f9-38ea-4cf9-8125-6bc3e93af154 ERROR
0f25e8cb-45f6-4f7f-83b4-4525dc245b11 FINISHED

SOAP equivalent:

wsclient -m finished -e http://example.com:8091/

To cancel a job, send a request to its URL with the DELETE verb.

curl -X DELETE  http://example.com/myproject/download/wss/jobs/093fd32b-4604-4c06-9ba9-d3aae7a6b80d

SOAP equivalent:

wsclient -m delete -e http://example.com:8091/ -j 093fd32b-4604-4c06-9ba9-d3aae7a6b80d

A job can be submitted to the Bridge by POSTing a job object to the jobs entity. Currently, there are two ways to do this; either by posting form-data (Content-Type: application/x-www-form-urlencoded) or by posting a JSON object (application/json). The following examples show how this can be achieved using curl.

In both cases, by default, after a successful submission, the Bridge will respond with a redirect header to the newly created jobs (Location: http://example.com/myproject/download/wss/jobs/<new_uuid>). The -L switch tells curl to follow this redirection.

Another possibility is to specify 'redir=0' in the URL (…/wss/jobs?redir=0); in this case, no redirection occurs, only the unique id of the new job is printed. This is the same behaviour as that of the SOAP interface.

To submit a job, the same information has to be supplied as in case of the SOAP interface. Notice that—unlike the SOAP interface—the URL, MD5 and size parts of a specification are specified separately, not in a single string joined with '=' .

Submitting a job using CURL

curl \
    -d name=dsp \
    -d grid=NULL \
    -d args='-dpf=docking.dpf' \
    -d tag='Test job' \
    -d input[docking.dpf][url]=http://example.com/myproject/download/inputs/publicautodock423/docking.dpf \
    -d input[docking.dpf][md5]=a7bac5a941a85660cd87bf83f93b852a \
    -d input[docking.dpf][size]=2933 \
    -d input[inputs.zip][url]=http://example.com/myproject/download/inputs/publicautodock423/inputs.zip \
    -d input[inputs.zip][md5]=cbb8c0ada8e197d6db707e6ce4f03ca6 \
    -d input[inputs.zip][size]=3270896 \
    -d output[0]=log.dlg \
    http://example.com/myproject/download/wss/jobs?hdr=1 -L

id grid alg status gridid args griddata tag creation_time metajobid userid error_info
022a9b75-8a08-4f5d-8188-c8687659e60d NULL dsp INIT  -dpf=docking.dpf  Test job 2013-01-22 13:30:32   

Alternatively:

Submitting a job using CURL

curl \
    # [ same as above ] \
    http://example.com/myproject/download/wss/jobs?redir=0

022a9b75-8a08-4f5d-8188-c8687659e60d

This is an example job description in JSON:

example-job.json

[
    {
        "name": "dsp",
        "grid": "NULL",
        "args": "-dpf=docking.dpf",
        "tag": "Test job",
        "input": {
            "docking.dpf": {
                "url": "http://example.com/myproject/download/inputs/publicautodock423/docking.dpf",
                "md5": "a7bac5a941a85660cd87bf83f93b852a",
                "size": 2933
            },
            "inputs.zip": {
                "url": "http://example.com/myproject/download/inputs/publicautodock423/inputs.zip",
                "md5": "cbb8c0ada8e197d6db707e6ce4f03ca6",
                "size": 3270896
            }
        },
        "output": [
            "log.dlg"
        ]
    }
]

This example job can be submitted with the following command.

Submitting a JSON job

curl \
    --header 'Content-Type: application/json' \
    --data @example-job.json \
    http://example.com/myproject/download/wss/jobs

[{"id":"98cba344-80cd-451a-bca8-cd3c887d0731"}]

If plain text output is required

curl \
    --header 'Content-Type: application/json' \
    --data @example-job.json \
    http://example.com/myproject/download/wss/jobs?format=plain

af3b0de8-c2cc-43eb-ba27-67ae1cda27db

Follow these steps to deploy the REST interface for the Bridge:

1. Create a directory that will be published by the project web server. For example, if you create a directory named wss in project/download, the directory will be available through http://<projecturl>/download/wss.
2. Setup the project site to allow RewriteEngine and security overrides in .htaccess files.
3. Allow write access to the Bridge output directory.
4. Setup security.
5. Copy the php scripts that constitute the interface to the directory just created.
6. Configure the interface by setting the necessary values in 3gb.php.

mkdir $HOME/project/download/wss

Setup Apache and the project site to support the Rewrite module and security overrides.

Enabling the Rewrite engine as ''root''

sudo a2enmod rewrite

In the project site configuration ([project_home]/project/[project_name].httpd.conf), FileInfo and Limit has to be added to AllowOverride for directory …/user. This enables the use of RewriteEngine and security overrides in .htaccess.

As output directories will be created upon submission, PHP must be allowed to access the 3G Bridge output directory (as configured in 3gbridge.conf/[wssubmitter]/output-dir). To do so, add this directory, delimited with a colon, to the open_basedir PHP admin value.

[project_home]/project/[project_name].httpd.conf

[...]
<Directory "[project_home]/project/html/user">
   [...]
    AllowOverride AuthConfig FileInfo Limit
   [...]
    php_admin_value "open_basedir"    "...:[project_home]/3g-bridge/output"
[...]   

Restart Apache for these changes to take effect:

sudo service apache2 restart

Assuming the Bridge output directory is [project_home]/master/3g-bridge/output. The actual value can be found in the 3G Bridge configuration file, in the [wssubmitter] section.

Make sure that the Apache process user (usually www-data) is added to the project user's default group

getent group $USER

boinc-home:x:108:www-data

If www-data is not listed next to the project user default group, you must correct this:

sudo usermod -a -G $USER www-data

When done, add group write access to the output directory and its direct descendants:

Make the output directories writable to the Apache user

cd $HOME/master/3g-bridge
chmod 775 `find output -maxdepth 1 -type d`

Create a logfile, which is writable by the Apache user:

Create a log file writable by the Apache user

LOGFILE="$HOME/project/log_<dir>/restinterface.log"
touch "$LOGFILE"
chmod g+w "$LOGFILE"

The following is an example .htaccess file we use, which enables the use of “pretty” URLs, and restricts access to our internal network. You can setup any kind of security Apache supports.

[project_home]/project/download/wss/.htaccess

Order deny,allow
Deny from all
Allow from lpds.sztaki.hu
 
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /[project_name]/download/wss/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ 3gb.php/$1 [L]
</IfModule>

To configure the interface, set the following values in config.inc.php. Usually, this involves only the following steps: (1) create a hard link to the Bridge configuration in the wss directory, (2) set the base URL of the REST interface, and (3) set the path of the log file. For completeness, other possibilities are also detailed here.

Set the base URL of the web service. It is required for redirects.

Set the path of the log file. This file must be writable by the Apache user.

Set the log level. Only messages with level at least this high will be logged. Possible settings—in order, highest to lowest—are:

1. CRITICAL – unrecoverable errors, unhandled exceptions.
2. ERROR – handled, but notable errors.
3. AUDIT – produces exactly two entries for each request:
   1. For security reasons, the fact that a request is being handled, including information on the client.
   2. A single line summarising what happened.
4. INFO – Detailed information on what happened.
5. DEBUG – Debug information useful for developers.
6. TRACE – Debug information useful for nobody.

For security reasons, we advise you to never set the level higher than AUDIT. Lower settings may produce extremely large amount of log data.

The CONFIG_FILE constant must point to the 3G Bridge config file. If Apache does not allow to follow symlinks or to access files anywhere on the filesystem, create a hard link to it in the wss dir.

Create a hard link to the 3G Bridge config file

cd [project_home]/project/download/wss
ln [project_home]/master/3g-bridge/3g-bridge.conf

The Bridge version is queried by executing BRIDGE_VER_CMD. Make sure the path it uses points to the 3g-bridge binary. The default setting is usually right.

Set the destination grid for meta-jobs (the name of the grid defined in 3g-bridge.conf). The default value is usually right.

Example: [project_home]/project/download/wss/config.inc.php

define('BASE_URL', 'https://PROJECT_URL/download/wss');
define('LOG_FILE', '/var/lib/boinc/home/project/log_home/restif.log');
define('LOG_LEVEL', 'AUDIT');
define('CONFIG_FILE', '3g-bridge.conf');
define('BRIDGE_VER_CMD',
       '/bin/bash -c /usr/sbin/3g-bridge\ -V');
define('METAJOB_GRID', 'Metajob');