The Generic BOINC Application Client (GBAC) is a virtualization enabled wrapper application. Beyond its name GBAC also represents a generic framework providing virtualized environments for various distributed computing infrastructures (DCIs). GBAC was implemented using the DC-API Meta API and does not rely on any middleware specific functionalities, thus it is possible to use it on any DCIs that are supported by DC-API, currently BOINC, Condor and XtremWeb. In this manual we detail the BOINC related aspects such as (i) deployment, (ii) configuration, (iii) usage and (iv) known limitations and workarounds.

This document will not detail the server side (BOINC project) related requirements, as they are no different than for any other BOINC application. On the client side GBAC has the following requirements:

• A PC with virtualization capable INTEL/AMD CPU
• At least 1GB RAM
• At least 2GB free disk space for the BOINC Client (considering the size of our sample virtual appliance and a typical application with inputs)
• Windows XP, Windows 7, Linux (or Mac OS X)
• VirtualBox 4.1.8+ installed
• BOINC Client 6.12.34+ installed

The following components are available for download in the downloads section at http://gbac.sourceforge.net

• Precompiled GBAC binaries,
• virtual appliances,
• and OPTIONAL GBAC guest extensions for custom virtual appliances.

An overview of GBAC can be found at the Components - GBAC page.

In the following sections we detail the components of GBAC.

The GBAC wrapper is a BOINC application handling the communication with the BOINC client and managing VirtualBox and the data transfer between host and guest. It is based on the VBoxWrapper of BOINC¹⁾.

The wrapper first creates a shared directory for file transfers between the host and the virtual machine. It copies all legacy application input files in this directory. GBAC does not separate the binaries of the legacy application and its input files. This means that all legacy application binaries and their input files are normal input files for GBAC and are not part of the GBAC BOINC application. As a next step the virtual machine is started using VirtualBox. GBAC does not contain and does not deploy VirtualBox; it is a prerequisite and every host must have VirtualBox preinstalled before GBAC can be used. Next, the legacy application is executed in the virtual machine by the GBAC guest extensions and the outputs are copied to the shared directory. Once the application finished the GBAC guest extensions shut down the virtual machine. Once the virtual machine terminates GBAC copies the results from the shared directory to the work directory of the BOINC client and terminates itself. The behavior of GBAC is configured via its configuration file, which we detail next.

GBAC requires the configuration file to have the BOINC logical name of vbox_job.xml (see Section Deployment for more details). As its name suggests it uses an XML-like sytax, below is shown an example. All parameters of the file must use the <foo>value</foo> or <bar/> syntax and the file must begin with <vbox_job_desc> and end with </vbox_job_desc>. The following parameters can be used:

• os_name: OS type to be used. A complete list can be obtained by running the VBoxManage list ostypes command. Usually set here Linux26, Debian for 32bit Linux and Linux26_64 or Debian_64 for 64bit Linux installed in the virtual appliance.
• memory_size_mb: Size of memory in MB to be allocated for the VM.
• image_filename: the logical filename of the virtual appliance GBAC will use. NOTE: The filname can end with .gz, in this case GBAC will ungzip it to the current working folder before the VM is created.
• enable_shared_directory: Must be always defined, or filetransfer between guest and host is not possible.
• enable_network: OPTIONAL. Enable network access for the guest. To disable network access remove the line from vbox_job.xml. By default it is disabled. The VA must support network (i.e., DHCP client must be available and enabled) to make use of this feature.

vbox_job.xml-example

<vbox_job_desc> 
  <os_name>Linux26</os_name> 
  <memory_size_mb>512</memory_size_mb> 
  <image_filename>gbac-debian-x86.vdi.gz</image_filename> 
  <enable_shared_directory/>
  <enable_network/> 
</vbox_job_desc>

Instead of listing legacy application binaries separately as input files they can be bundled in an archive and only the archive needs to be specified. The following rules should be used when creating an application bundle:

• Input files of the legacy application should not be included in the bundle'.
• Either ZIP (and .zip extension) or TAR and GZIP (and .tar.gz or .tgz extension) utilities can be used.
• The bundle should have the <EXECUTABLE NAME>.app.<EXTENSION> logical filename, where

<EXECUTABLE NAME> is the filename of the main executable in the bundle and <EXTENSION> is the extension depending on the archiver used, e.g., if the main executable is my_executable and ZIP is used then the bundle must be named my_executable.app.zip.

The essence of GBAC is a virtual appliance which contains an operating system, the VirtualBox guest extensions and the GBAC guest extensions. It is used by VirtualBox to boot a virtual machine which in turn is used to execute legacy applications, see Section Custom virtual appliances for more details on the contents of the VA.

The filesystem of the virtual machine is reseted to its original state for each work unit (it uses the immutable option of VirtualBox). Thus no work unit can do any permanent harm to it. The virtual appliance is part of the GBAC application (and not of the legacy application) and it is downloaded by any client only once.

We provide a Debian Linux 6.0 32bit virtual appliance which is able to execute most applications and also can be used as basis for custom virtual appliances. The console login information for the virtual appliance is the following:

• user name: root
• password: abc123. Note: the period at the end is part of the password

The virtual appliance for GBAC contains several additions and modifications compared to a standard Linux installation:

• The VirtualBox guest additions (for Linux) installed
• Init script for the GBAC guest addition: /etc/init.d/gbac
• The client script which manages the application in the virtual machine: /root/runme.sh
• A guest-host shared directory named /mnt/shared/ which is mounted during boot time.

The GBAC guest extensions are responsible for (a) mounting the guest-host shared directory, (b) copying the legacy application files to the sandbox or extracting the legacy application bundle (see Section Legacy application bundle) to the sandbox and ( c) executing the legacy application.

The functionality of GBAC is detailed at the Components - GBAC page.

In the following subsections we detail (i) the required steps to deploy GBAC on the server and (ii) configuration for the clients: we are going to deploy GBAC in a BOINC project, configure a BOINC Client and submit and execute an example GBAC job.

We assume the following:

• The BOINC project was created using the SZDG Debian packages. More information about the packages and BOINC project set up is available at ²⁾.
• A BOINC project is already set up, configured and available at the following address: http://example.com/gbacsample/
• The name of the project is gbacsample.
• The home directory of the BOINC project on the file system is (this is the home directory of the BOINC project admin user).
• The root directory of BOINC project on the file system is (this directory is usually the /project/ directory).

In the next sections we describe two methods for deploying GBAC. First (in section Deployment with boinc_appmgr) we show how to do the deployment via the boinc_appmgr tool provided by the SZDG package, and in section Manual deployment we describe how to deploy GBAC manually on the server.

Easiest way to deploy GBAC is with the boinc_appmgr tool. There are three .xml files present in the GBAC binary bundle as shown in the next listing with the <PLATFORM> part replaced by the current application bundle platform name (e.g., i686-pc-linux-gnu or windows_intelx86):

  gbac-<PLATFORM>-client-debug.xml
  gbac-<PLATFORM>-client.xml 
  gbac-master.xml

These files are passed as parameters (detailed later in this section) for boinc_appmgr, and are for the following:

• gbac-<PLATFORM>-client-debug.xml - Deploy the GBAC BOINC application with debugging enabled (see section sec:debug for more details on debugging).
• gbac-<PLATFORM>-client.xml - Deploy the GBAC BOINC application without debugging enabled.
• gbac-master.xml - Deploy GBAC server side components, i.e. the validator (sample_trivial_validator).

From these .xml files you need one client file (either gbac-<PLATFORM>-client.xml or gbac-client-debug.xml) and the server file. To deploy GBAC perform the following steps and execute the following commands as the project administrator user:

1. Deploy the GBAC BOINC client application
   • without debugging enabled:

     boinc_appmgr --add --client gbac-<PLATFORM>-client.xml

   • with debugging:

     boinc_appmgr --add --client gbac-<PLATFORM>-client-debug.xml

2. Deloy the GBAC server components (i.e. validator):

   boinc_appmgr --add --master gbac-master.xml

In the following we will deploy GBAC for Linux 64bit. GBAC is a BOINC application so the procedure contains no different steps than for any other BOINC application, but still there are some specific requirements which we want to detail here.

1. A new application and application version must be created. Open project.xml of the project and add the following contents to the contents, right before the closing </boinc>:

     <app> 
       <name>gbac</name> 
       <user_friendly_name>GBAC</user_friendly_name> 
     </app> 

   It should look like this afterwards:

     [...]
     <app> 
       <name>gbac</name> 
       <user_friendly_name>GBAC</user_friendly_name> 
     </app> 
   </boinc>

2. Deploy GBAC binaries at the BOINC project.
   • Create the directory structure gbac/gbac_1.80_x86_64-pc-linux-gnu inside /apps/ for GBAC. The first part (gbac/) is for the application and the second part inside the first one (gbac_1.80_x86_64-pc-linux-gnu) is for the current version (1.80) of the Linux 64 bit application (x86_64-pc-linux-gnu).
3. The GBAC files should be copied to the directory created previously with some additional files. The directory should contain the following files:

   gbac_1.80_x86_64-pc-linux-gnu 
   gbac_1.80_x86_64-pc-linux-gnu-debian60-x86.vdi.gz 
   gbac_1.80_x86_64-pc-linux-gnu-debian60-x86.vdi.gz.file_ref_info 
   gbac_1.80_x86_64-pc-linux-gnu-vbox_job.xml 
   gbac_1.80_x86_64-pc-linux-gnu-vbox_job.xml.file_ref_info

   • The GBAC wrapper binary renamed to gbac_1.80_x86_64-pc-linux-gnu.
   • The compressed virtual appliance renamed to gbac_1.80_x86_64-pc-linux-gnu-debian60-x86.vdi.gz.
   • An info file for the VA named gbac_1.80_x86_64-pc-linux-gnu-debian60-x86.vdi.gz.file_ref_info with the following contents:

     <copy_file> 
     <open_name>gbac-debian-x86.vdi.gz</open_name>

   • The configuration file for GBAC named gbac_1.80_x86_64-pc-linux-gnu-vbox_job.xml with the following contents:

     <vbox_job_desc> 
      <os_name>Linux26</os_name> 
      <memory_size_mb>512</memory_size_mb>
      <image_filename>debian60-x86.vdi</image_filename> 
      <enable_shared_directory/>
     </vbox_job_desc>

   • An info file for the configuration file named gbac_1.80_x86_64-pc-linux-gnu-vbox_job.xml.file_ref_info with the following contents:

     <copy_file> 
     <open_name>vbox_job.xml</open_name>

4. Register the application and application version by running the xadd and update_versions commands.
5. Install a validator for GBAC. The sample_trivial_validator supplied by BOINC should be used:
   • Open config.xml and add the following contents to the daemons section of the file:

     <daemon> 
      <cmd>sample_trivial_validator -d 3 -app gbac</cmd>
     </daemon>

   • restart the project by issuing the stop and start commands after each other.

The wsclient utilility of 3G Bridge will be used for task submission in Section sec:usecase., but first 3G Bridge has to be configured for GBAC. We assume that it is already deployed and configured for the BOINC project through the DC-API-Single plug in and only the GBAC related configuration is missing. See the 3g-bridge manual for more information and examples how to deploy 3G Bridge on a BOINC project. The following steps should be performed for GBAC:

1. Open the /master/3g-bridge/dc-api.conf and add the following contents to the end of the file:

   [Client-gbac]
   MaxOutputSize = 210485760
   MaxDiskUsage = 2048000000
   MaxMemUsage = 768MiB
   EnableSuspend = false
   MinQuorum = 1
   TargetNResults = 1
   MaxErrorResults = 2
   MaxTotalResults = 5
   MaxSuccessResults = 5
   MaxFPOps = 1.0e+16

   The options set here are detailed in the DC-API manual³⁾ in section 'Per-client configuration'. The parameter MaxFPOps determines maximum floating-point operations and thus the run time limit for workunits. Increase this (and restart the project) when GBAC is terminated by the BOINC Client because it reached this limit. Same applies for MaxOutputSize, MaxDiskUsage and MaxMemUsage.</p>

2. Add GBAC to the cg_algqueue table of 3G Bridge database by executing the following commands as the BOINC project user:

    mysql boinc_gbacsample 

   Here gbacsample is the short name of the BOINC project.

    insert into cg_algqueue (grid, alg, batchsize, statistics) \
       values ('DG', 'gbac', 1, NULL);

   Here we assume that the BOINC project (and the DC-API-Single plug in) is configured as DG grid in 3G Bridge.

3. Restart 3G Bridge by restarting the BOINC project with issuing the stop and start commands after each other.

GBAC requires the BOINC Client and VirtualBox to be installed on all resources. Installing VirtualBox is a straightforward procedure which we don't detail here. Installing and configuring BOINC for GBAC is also simple, but there are some specific steps required which we detail here.

1. During install make sure that Protected application execution is not selected (see next figure). If you need the Protected application execution option please see BOINC Client on Windows: sandboxed and service installation for more details.

   

   BOINC install dialog on Windows

2. After the BOINC Manager started select Advanced View (see figures below).
3. In Advanced View select the Run always option of the Activity option (see next figure). This is required as a workaround for a bug in the current version, see Non-BOINC workload too high and client suspends computation for more details.
4. Finally connect the Client to the BOINC Project where GBAC is deployed.

In this example the wsclient command line utility will be used to (a) submit a legacy application to BOINC for the GBAC application, (b) monitor the status of the execution and © retrieve outputs once finished. We assume the following:

• The ws-submitter component of 3G Bridge is configured and running on http://example.com:9001/ . See the 3g-bridge manual for more information and examples how to set up ws-submitter for 3G Bridge.
• The EMMIL⁴⁾ application will be used for submission. The application bundle (zip) and sample input file can be downloaded from here: https://sourceforge.net/projects/gbac/files/ .
• The EMMIL bundle and input files are downloaded to the /tmp/emmil/ directory.

The following steps need to be performed:

• Submit the EMMIL application and its input file as a work unit for GBAC. It can be done via the following commands:

• FILESDIR=/tmp/emmil/ 
  wsclient -repeat 1 -m add -e http://example.com:9001 \
           -g DG -a wu_script.sh -n gbac \
           -i JobInputReal.txt=$FILESDIR/JobInputReal.txt \
           -i emmil_i686-pc-linux-gnu.app.tar.gz=$FILESDIR/emmil_i686-pc-linux-gnu.app.tar.gz   
           -o LPSolvOutput.txt 

• This wsclient command will print an UUID for the submitted task:

• b82ca283-e9c1-49bf-98be-335adc4b4252

• This UUID can be used for querying the status of the task and retrieving its outputs once it has finished. The command line options used for wsclientare explained in the wsclient manual page of the wsclient Debian package (man wsclient).

• The status of the task can be queried using the UUID and wsclient:

• wsclient -m status -e 'http://example.com:9001' \
            -j b82ca283-e9c1-49bf-98be-335adc4b4252

• This will give the following result:

• b82ca283-e9c1-49bf-98be-335adc4b4252 Running

• Once the task has finished (the reported status is 'Finished' instead of 'Running') the location of its ouptuts can be queried using the following command and retrieved from the printed URLs e.g., via wget:

• wsclient -m output -e 'http://example.com:9001' \
           -j b82ca283-e9c1-49bf-98be-335adc4b4252

• Result of GBAC job status query:

• Output files for job 'b82ca283-e9c1-49bf-98be-335adc4b4252': 
     LPSolvOutput.txt http://example.com/3g-bridge-gbacsample/b8/b82ca283-e9c1-49bf-98be-335adc4b4252/LPSolvOutput.txt

GBAC allows to pass environment variables to the job/ application executed in the VM. For the job a separate environment will be set up and the variables will be exported there. GBAC has a gbac_job.xml configuration file that that serves two purposes:

1. The main executable and command line can be defined here as an alternative to the <MAIN_EXE>.app.tar.gz app bundle format
2. Environment variables can be passed to the application.

The format of gbac_job.xml is as follows:

gbac_job.xml-example

<gbac_job>
  <setenv>FOO="foo"</setenv>
  <setenv>BAR="bar"</setenv>
  <main_executable>emmil_i686-pc-linux-gnu</main_executable>
</gbac_job>

• It may contain multiple <setenv>VARIABLE=“VALUE”</setenv> statements where the VARIABLE with value VALUE will be exported for the application in the VM.
• It may contain a single <main_executable><MAIN_EXE></main_executable> line defining the main executable (MAIN_EXE) of the application.
• The application bundle (*.app.tar.gz) can not contain the gbac_job.xml, it must be separate input file for the job.

It is possible to create custom virtual appliances for GBAC containing custom operating systems, libraries and 3rd party tools, but the following should be considered:

• Any operating system, library or piece of software can be used, but open source licensed ones are preferred, otherwise each host where GBAC job is running requires licenses.
• Each virtual appliance must contain the VirtualBox guest extensions installed otherwise it is not possible to transfer files between guest and host.
• Each virtual appliance must contain the GBAC guest extensions (see Section GBAC guest extensions).

In the following section we detail the steps for deploying GBAC Guest Tools in the CernVM VA, that is a Scientific Linux (Red Hat) based image.

The following steps needed to deploy GBAC Guest Tools in the CernVM VA:

1. Install VirtualBox Guest Additions
2. Create a user named ‘vmsandbox' with its home in /home/vmsandbox/:

   adduser vmsandbox

3. Add vmsandbox to the wheel group:

   usermod -a -G wheel vmsandbox

4. Create the shared mount directory:

   mkdir -p /mnt/shared

5. Create a log directory inside the vmsandbox users home directory:

   mkdir -p /home/vmsandbox/logs

6. Note the user and group id of vmsandbox user, by running the ‘id vmsandbox’ command. The output should be similar to this:

   $>id vmsandbox
   uid=500(vmsandbox) gid=501(vmsandbox) groups=10(wheel),501(vmsandbox)

   The number after uid= is the user id (500 in this case), and the number after gid= is the group id (501 in this case).

7. Add the following line to /etc/fstab where the number after gid= is the group id and the number after uid is the user id:

   shared     /mnt/shared     vboxsf     gid=501,uid=500     0 0

8. Copy the GBAC guest tools to the VM (Here it is assumed the shared directory ‘shared’ contains these files. Add a share named ‘shared’ to the vm in VirtualBox and reboot it):

   cp /mnt/shared/gbac /etc/init.d/
   cp /mnt/shared/guest-tools.sh /home/vmsandbox

9. Add gbac script to chkconfig:

   chkconfig --add gbac

10. Enable gbac script for runlevel 3 (and disable for runlevel 2):

    chkconfig —level 2 gbac off
    chkconfig —level 3 gbac on

11. Add the following to the END of /etc/sudoers (make sure sudoers is installed):

    Defaults:vmsandbox        !requiretty
    vmsandbox ALL = (root) NOPASSWD: /sbin/shutdown -h now

GBAC provides several features that ease application and virtual appliance development and debugging. In the following we detail these.

In this section we detail the debug configuration file of GBAC and its parameters. GBAC requires the configuration file to have the BOINC logical name of vbox_debug.xml. As the name suggests it uses an XML-like sytax, The next code block shows an example. All parameters of the file must use the <foo>value</foo> or <bar/> syntax and the file must begin with <vbox_debug> and end with </vbox_debug>.

<vbox_debug> 
  <display_gui/> 
  <log_vboxmanage/> 
  <not_standalone/> 
</vbox_debug>

The following parameters can be used:

• 'display_gui': Switch between headless and GUI modes (default: headless): by default the virtual machine is run in the background without any display window. When this parameter is set a window is used as the display of the virtual machine.
• 'log_vboxmanage': Log vboxmanage commands and their results to stderr as well (default: no).
• 'not_standalone': For vm name generation debugging, assume false where standalone mode is checked in the wrapper code.

The virtual machine started by GBAC will shutdown automatically when the task finished (or if no task can be started). For debugging purposes it is possible to disable this behavior: if a file named no_shutdown is present in the guest-host shared directory the virtual machine will keep running and can be shut down only by hand (either from the console or from the VirtualBox menu).

To run the GBAC VA as a “standalone” virtual machine do the following steps:

1. Create a new virtual machine in VirtualBox
2. Add a SATA controller and bind the VA to Port 0
3. Add a shared folder named shared with Full access to the virtual machine. Make sure that the automount option is not selected!
4. Place a file named no_shutdown into this shared folder.
5. Optional: Enable networking (e.g., NAT). It will be detected and the network interface enabled.
6. Start up the virtual machine. It won't shut itself down automatically. (The root password is 'abc123.')

Place a file named display_gui in the working directory of GBAC before the VM is started to start VirtualBox in windowed mode (same as the display_gui option of vbox_debug.xml).

There are no networking related components (i.e., ifconfig, dhclient) in the default VA. To install additional components via network the following steps must be performed:

1. Create a virtual machine in VirtualBox using the image. Add a shared folder named shared and put a file named no_shutdown in it. This will make the VM run endlessly (else the guest components would shut down the machine after the - failed - execution of the non existent application). Configure network for the VM in VirtualBox.
2. Download the following packages and put them in the shared directory on the host:
   • http://packages.debian.org/squeeze/i386/net-tools/download
   • http://packages.debian.org/squeeze/i386/isc-dhcp-client/download
   • http://packages.debian.org/squeeze/i386/isc-dhcp-common/download
   • http://packages.debian.org/squeeze/i386/iproute/download
3. From now on all steps must be executed inside the virtual machine.
4. Run the followin in the shared directory (/mnt/shared), this will install all packages:

   dpkg -i *.deb

5. Run the following:

   dmesg | grep eth

   this should print something similar (instead of eth2 can be anything):

   [...] e1000: eth0 [...] 
   [...] renamed network interface eth0 to eth2 

6. Run the following (instead of eth2 the equivalent you saw in the previous query):

   ifconfig eth2 up

7. Run the following (instead of eth2 the equivalent you saw in the previous query) and you should have networking:

   dhclient eth2

8. You can use apt-get install … to install additional packages.

GBAC provides 3 log files when requested:

• Standard error (gbac-app.stdout),
• Standard output (gbac-app.stderr) and
• a full debug log (gbac-exec.log)

These files should be specified as output files e.g., with wsclient (see code below) and will be automatically generated by GBAC.

 wsclient \
 [...]
 -o gbac-app.stdout \
 -o gbac-app.stderr \
 -o gbac-exec.log

In the following subsections we detail the known limitations of GBAC and the proposed workarounds and resolutions. The limitations are marked with (L) and the resolutions with (R).

(L) The running VirtualBox VM is a separate process that BOINC does not consider its own. This means that if the client has a setting that it should suspend computation if the non-BOINC CPU load goes above a threshold, it will constantly suspend-resume the virtual machine.

(R) Currently the only workaround is to set BOINC to always run in the BOINC Manager. We'll fix this in an upcoming release…

(L) We found that BOINC has sometimes trouble detecting VirtualBox (registry virtualization), thus GBAC is also not able to detect it and will permanently fail.

(R) We are investigating this issue, and still don't have any workaround for it.

(L) On Windows if the BOINC Client is installed and run under a restricted account (when installed with Protected application execution option) it won't have privileges to start a virtual machine via VirtualBox.

(R) To fix this the following modifications have to be done (as administrator):

1. The BOINC installer created two user accounts: boinc_admin (or boinc_master) and boinc_project.
2. The boinc_project account needs to have full control on the following two directories (adjust path if not installed on the C drive or in the default location):
   • C:\Program Files\Oracle\VirtualBox\
   • C:\WINDOWS\system32\config\systemprofile\.VirtualBox\ – If this directory does not exist, create it and grant the privileges.
3. The boinc_project account also needs to have Launch and Activation, Access and Configuration Permissions on the VirtualBox DCOM service.
   1. Go to Control Panel → Administrative Tools → Component Services → Computers → My Computer → DCOM Config → VirtualBox Application
   2. Right click 'properties', select the 'Security' tab.
   3. For each section select custom → Browse for local user → assign all as allow. Repeat for each custom setting.

(L) Currently it is not possible to cache the legacy binaries on clients and hence they will be downloaded for each time for every work unit.

(R) We are aware of this problem and working on it for a near future release.

(L) Only generic validators (e.g., sample trivial validator) can be used with GBAC.

(R) GBAC is a generic framework for executing legacy applications, thus it is not possible to use application specific validators.

Please report bugs and provide feedback via email to desktopgrid AT lpds DOT sztaki DOT hu.