- Simple scenarios
- Complex scenarios
- Bridging scenarios
- Basic components
- Application support
- Job submission interface alternatives
- Additional components
- Basic components
- Application support
- Additional components
Table of Contents
3G Bridge Installation
This manual describes the installation and configuration of the 3G Bridge with BOINC integration (DC-API).
The 3G Bridge and BOINC have been developed for Debian 6 systems. Although the Brigde is known to work with Ubuntu and Scientific Linux 6, these systems are different from Debian. Ubuntu is only slightly different, and most of the scripts listed here will work as-is, or with little modification. In SL6, however, the scripts and the list of packages needed to be installed are very different, and are not covered here.
To use the 3G Bridge as a front-end for BOINC, a working BOINC project has to be created before the Bridge can be configured.
The Bridge packages can be found in the SZDG repositories. Also, the Bridge uses DC-API as an interface to BOINC, which is also found in this repository. Whether you are installing the Bridge from packages or compiling the Bridge from source, you will have to have the SZDG repository in your APT sources list.
# As root echo 'deb http://www.desktopgrid.hu/debian/ squeeze szdg' >> /etc/apt/sources.list apt-key advanced --keyserver keys.niif.hu --recv-keys 6A2165907B1AAC6F apt-get update
The 3G Bridge can be easily installed from packages. The installer will ask whether you want to use the Bridge in standalone mode. As you will use the Bridge with BOINC, Select NO.
wsclientis not mandatory on the server, but it is useful for testing job submission.
# As root apt-get install 3g-bridge 3g-bridge-wsclient
The Bridge can be installed from source if absolutely necessary, but installing the Bridge from the package repository is the preferred way.
Integrating the 3G Bridge with BOINC
At this point, there is a working BOINC project on the server, and the 3G Bridge has been installed and is ready to be used as a BOINC master application. We assume that 1) the Bridge has been installed from packages, and 2) the project is called
test. When using this manual, don't forget to change the BOINC username, the database name, etc. to match your actual project name.
Preparing the Database
Switch to the BOINC project user.
To use the Bridge as a master application, you must first create the 3G Bridge database. You can simply create the 3G Bridge tables in the already existing BOINC database 1). Provide username and password to mysql as necessary.
export PNAME=test # project name sudo su - boinc-$PNAME stop # don't forget to stop the project first mysql boinc_$PNAME < /usr/share/3g-bridge/schema.sql
Creating Job Queues
If you have no client applications installed in BOINC yet, you can skip this step for now.
You can add or remove queues any time in the future, provided you stop the project first.
Each queue has two defining attributes:
gridA single Bridge can be the entry point for multiple back-end infrastructures—the
gridattribute refers to one of these back-ends. The value of this attribute must match one of the queue definitions in the 3G Bridge configuration.
algThe name of the application in the
grid. The value of this attribute must match an installed BOINC application's name.
algattributes together define a single queue. You have to define a queue for each application registered in BOINC.
The 3rd attribute of a queue (
batchsize) defines the number of jobs the Bridge will process in a single iteration. The larger this value is, the better the throughput of the 3G Bridge will be for a single queue. On the other hand, a larger
batchsizewill consume more memory, may increase the load of the system, and, if you have many queues, it will increase the wait time of other queues. Generally, 10 is an acceptable default value for this attribute.
mysql boinc_$PNAME -e \ "insert into cg_algqueue (grid, alg, batchsize) values ('gridname', 'algname', 10)"
Adding the 3G Bridge to BOINC
After creating the 3G Bridge database, you can add the Bridge to BOINC as a master application. A new directory will be created in the project home directory:
'$HOME/master/3g-bridge/'. This directory contains all Bridge configuration files, which will be covered in the next section.
boinc_appmgr --add --master /usr/share/3g-bridge/master-ws.xml
The configuration information of the Bridge is in the file
$HOMEis the project home directory. This file is a glib ini file, which defines key-value pairs grouped in sections. The configuration file and the values in it are case sensitive. The semantics of the configuration file is described in comments in the file and detailed in the 3G Bridge manual (
man 5 3g-bridge.conf). Here, we will only cover the necessary steps to initally configure the 3G Bridge.
The following listing shows an excerpt from a
3g-bridge.conffile. The keys included in this listing are those that must be set.
[database] host = localhost name = boinc_<projectname> user = boinc_<projectname> password = <found in $HOME/project/config.xml> [bridge] log-target = stdout pid-file = /var/lib/boinc/<projectname>/master/3g-bridge/bridge.pid [wsmonitor] log-target = stdout pid-file = /var/lib/boinc/<projectname>/master/3g-bridge/monitor.pid port = <8092> [wssubmitter] port = <8091> log-target = stdout pid-file = /var/lib/boinc/<projectname>/master/3g-bridge/wssubmitter.pid input-dir = /var/lib/boinc/<projectname>/master/3g-bridge/input output-dir = /var/lib/boinc/<projectname>/project/download/bridge-output output-url-prefix = http://<project_url>/download/bridge-output [<gridname>] handler = DC-API-Single dc-api-config = /var/lib/boinc/<projectname>/master/3g-bridge/dc-api.conf [dlmgr] handler=DownloadManager download-threads=10 ...
The following list details the modifications you have to make in the default configuration file. The list is not ordered, and it's very detailed; we recommend you to only skim through it before modifying the configuration file to know what you will be doing, and then use the listing above as a guide for modifying your configuration.
[defaults]section contains default and general setting, used by all or any component of the Bridge.
log-levelas you need to. It may be pratical to set it to
DEBUGfor a while after the installation.
monlog-*keys are detailed in the 3G Bridge monitoring page.
[database]section so the Brigde will be able to access the database. Typical values are shown in the listing, change them as necessary. The password can be found in
These settings apply to all components (active processes) of the Bridge:
log-targetsetting for all components should be
stdout. In this case, the Bridge logs will appear in the standard log directory of the project:
pid-filesetting for all components to a path that they can use. Set different file names for each component, and make sure the specified files can be createted and/or overwritten. See the listing for an example.
output-dirsettings must point to existing directories. So don't forget to create these directories, if they don't exist. The Bridge must have write permission on both directories.
INDIR=/var/lib/boinc/<projectname>/master/3g-bridge/input OUTDIR=/var/lib/boinc/<projectname>/project/download/bridge-output mkdir -p "$INDIR" "$OUTDIR"
The output directory must be accessible through http. (It's usually a good practice to place the output directory in
$HOME/project/download.) Set the value of
output-url-prefixto the http URL through which the output directory is accessible. This is usually
If you have multiple projects, don't forget to set a separate port for each one. This can be any free port in the non-privileged range (>1024). If you have a single project and no other services running on the host, you can leave all ports with their default values.
Configuring plugins (queues)
Configure the queues of the 3G Bridge. Each additional section will configure a queue for the Bridge. The name of the section will be the name of the queue, which has to match the queue name specified in
disablesetting of a queue to
truecompletely disables it. Example queues in the default configuration are disabled by default, you can just ignore them.
handlersetting for a queue will define the plugin which handles that queue. Other configuration options in the plugin's section depend on which handler is configured.
Configuring the BOINC plugin
In case of BOINC, the plugin is
'DC-API-Singlmanual/3gbridge.txt · Last modified: 2013/02/27 13:29 (external edit)