User Tools

Site Tools


Sidebar

Overview

Navigation

Scenarios

Components

Manuals

  • Basic components
  • Application support
  • Additional components
    • Command disabled: revisions
      manual:3gbridge

      3G Bridge Installation

      This manual describes the installation and configuration of the 3G Bridge with BOINC integration (DC-API).

      Prerequisites

      The 3G Bridge and BOINC have been developed for Debian 6 systems. Although the Brigde is known to work with Ubuntu and Scientific Linux 6, these systems are different from Debian. Ubuntu is only slightly different, and most of the scripts listed here will work as-is, or with little modification. In SL6, however, the scripts and the list of packages needed to be installed are very different, and are not covered here.

      To use the 3G Bridge as a front-end for BOINC, a working BOINC project has to be created before the Bridge can be configured.

      The Bridge packages can be found in the SZDG repositories. Also, the Bridge uses DC-API as an interface to BOINC, which is also found in this repository. Whether you are installing the Bridge from packages or compiling the Bridge from source, you will have to have the SZDG repository in your APT sources list.

      # As root
      echo 'deb http://www.desktopgrid.hu/debian/ squeeze szdg' >> /etc/apt/sources.list
      apt-key advanced --keyserver keys.niif.hu --recv-keys 6A2165907B1AAC6F
      apt-get update

      Installation

      The 3G Bridge can be easily installed from packages. The installer will ask whether you want to use the Bridge in standalone mode. As you will use the Bridge with BOINC, Select NO.

      Installing the wsclient is not mandatory on the server, but it is useful for testing job submission.

      # As root
      apt-get install 3g-bridge 3g-bridge-wsclient

      The Bridge can be installed from source if absolutely necessary, but installing the Bridge from the package repository is the preferred way.

      Integrating the 3G Bridge with BOINC

      At this point, there is a working BOINC project on the server, and the 3G Bridge has been installed and is ready to be used as a BOINC master application. We assume that 1) the Bridge has been installed from packages, and 2) the project is called test. When using this manual, don't forget to change the BOINC username, the database name, etc. to match your actual project name.

      Preparing the Database

      Switch to the BOINC project user.

      To use the Bridge as a master application, you must first create the 3G Bridge database. You can simply create the 3G Bridge tables in the already existing BOINC database 1). Provide username and password to mysql as necessary.

      export PNAME=test        # project name
      sudo su - boinc-$PNAME
      stop                     # don't forget to stop the project first
      mysql boinc_$PNAME < /usr/share/3g-bridge/schema.sql

      Creating Job Queues

      If you have no client applications installed in BOINC yet, you can skip this step for now.

      You can add or remove queues any time in the future, provided you stop the project first.

      Each queue has two defining attributes:

      • grid A single Bridge can be the entry point for multiple back-end infrastructures—the grid attribute refers to one of these back-ends. The value of this attribute must match one of the queue definitions in the 3G Bridge configuration.
      • alg The name of the application in the grid. The value of this attribute must match an installed BOINC application's name.

      The grid and alg attributes together define a single queue. You have to define a queue for each application registered in BOINC.

      The 3rd attribute of a queue (batchsize) defines the number of jobs the Bridge will process in a single iteration. The larger this value is, the better the throughput of the 3G Bridge will be for a single queue. On the other hand, a larger batchsize will consume more memory, may increase the load of the system, and, if you have many queues, it will increase the wait time of other queues. Generally, 10 is an acceptable default value for this attribute.

      mysql boinc_$PNAME -e \
         "insert into cg_algqueue (grid, alg, batchsize) values ('gridname', 'algname', 10)"

      Adding the 3G Bridge to BOINC

      After creating the 3G Bridge database, you can add the Bridge to BOINC as a master application. A new directory will be created in the project home directory: '$HOME/master/3g-bridge/' . This directory contains all Bridge configuration files, which will be covered in the next section.

      boinc_appmgr --add --master /usr/share/3g-bridge/master-ws.xml

      Configuration

      The configuration information of the Bridge is in the file '$HOME/master/3g-bridge/3g-bridge.conf' , where $HOME is the project home directory. This file is a glib ini file, which defines key-value pairs grouped in sections. The configuration file and the values in it are case sensitive. The semantics of the configuration file is described in comments in the file and detailed in the 3G Bridge manual (man 5 3g-bridge.conf). Here, we will only cover the necessary steps to initally configure the 3G Bridge.

      The following listing shows an excerpt from a 3g-bridge.conf file. The keys included in this listing are those that must be set.

      $HOME/master/3g-bridge/3g-bridge.conf
      [database]
      host     = localhost
      name     = boinc_<projectname>
      user     = boinc_<projectname>
      password = <found in $HOME/project/config.xml>
       
      [bridge]
      log-target = stdout
      pid-file = /var/lib/boinc/<projectname>/master/3g-bridge/bridge.pid
       
      [wsmonitor]
      log-target = stdout
      pid-file = /var/lib/boinc/<projectname>/master/3g-bridge/monitor.pid
      port = <8092>
       
      [wssubmitter]
      port = <8091>
      log-target = stdout
      pid-file = /var/lib/boinc/<projectname>/master/3g-bridge/wssubmitter.pid
      input-dir = /var/lib/boinc/<projectname>/master/3g-bridge/input
      output-dir = /var/lib/boinc/<projectname>/project/download/bridge-output
      output-url-prefix = http://<project_url>/download/bridge-output
       
      [<gridname>]
      handler = DC-API-Single
      dc-api-config = /var/lib/boinc/<projectname>/master/3g-bridge/dc-api.conf
       
      [dlmgr]
      handler=DownloadManager
      download-threads=10
       
      ...

      The following list details the modifications you have to make in the default configuration file. The list is not ordered, and it's very detailed; we recommend you to only skim through it before modifying the configuration file to know what you will be doing, and then use the listing above as a guide for modifying your configuration.

      [defaults]

      The [defaults] section contains default and general setting, used by all or any component of the Bridge.

      Set the log-level as you need to. It may be pratical to set it to DEBUG for a while after the installation.

      The monlog-* keys are detailed in the 3G Bridge monitoring page.

      [database]

      Setup the [database] section so the Brigde will be able to access the database. Typical values are shown in the listing, change them as necessary. The password can be found in $HOME/project/config.xml or in $HOME/.my.cnf.

      All components

      These settings apply to all components (active processes) of the Bridge: wssubmitter, bridge, wsmonitor.

      The log-target setting for all components should be stdout. In this case, the Bridge logs will appear in the standard log directory of the project: '$HOME/project/log_<hostname>/app_3g-bridge_<componentname>.log' .

      Set the pid-file setting for all components to a path that they can use. Set different file names for each component, and make sure the specified files can be createted and/or overwritten. See the listing for an example.

      wssubmitter

      The input-dir and output-dir settings must point to existing directories. So don't forget to create these directories, if they don't exist. The Bridge must have write permission on both directories.

      INDIR=/var/lib/boinc/<projectname>/master/3g-bridge/input
      OUTDIR=/var/lib/boinc/<projectname>/project/download/bridge-output
       
      mkdir -p "$INDIR" "$OUTDIR"

      The output directory must be accessible through http. (It's usually a good practice to place the output directory in $HOME/project/download.) Set the value of output-url-prefix to the http URL through which the output directory is accessible. This is usually http://<project_url>/download/bridge-output.

      If you have multiple projects, don't forget to set a separate port for each one. This can be any free port in the non-privileged range (>1024). If you have a single project and no other services running on the host, you can leave all ports with their default values.

      Configuring plugins (queues)

      Configure the queues of the 3G Bridge. Each additional section will configure a queue for the Bridge. The name of the section will be the name of the queue, which has to match the queue name specified in cg_alqueue.

      Setting the disable setting of a queue to true completely disables it. Example queues in the default configuration are disabled by default, you can just ignore them.

      The handler setting for a queue will define the plugin which handles that queue. Other configuration options in the plugin's section depend on which handler is configured.

      Configuring the BOINC plugin

      In case of BOINC, the plugin is 'DC-API-Singl

      manual/3gbridge.txt · Last modified: 2013/02/27 13:29 (external edit)