GenWrapper

Introduction

GenWrapper offers a generic solution for wrapping and executing an arbitrary set of legacy applications in a BOINC infrastructure. The main strength of it is the POSIX like shell scripting environment which is used to describe how the application is to be run and how the work unit should be processed. This provides great flexibility and a powerful tool to port legacy applications to Desktop Grids with very little effort.

GenWrapper is based on GitBox which is a stripped down version of BusyBox that was ported to run on Windows. We ported GitBox back to UNIX (Linux and Mac OS X) and extended it with BOINC specific commands and a Launcher component that handles starting GitBox and communicating with the Core Client such as reporting CPU time and handling Suspend/Resume requests, much like the BOINC Wrapper does. Currently the GitBox and Launcher components of GenWrapper are two separate executables, each about 400 kiB size.

Overview: Applications under BOINC

Volunteers are joining the DG by installing and running the BOINC Core Client and attaching to one or more projects. Besides handling communication with the project servers, the Core Client is responsible for:

  • starting, stopping, suspending and resuming the application;
  • enforcing resource limits and resource shares between different projects set by the user;
  • instructing the application to checkpoint itself and
  • accepting various statistics reported by the application (its completion percentage, used CPU time).

To use the distributed resources gathered by BOINC, the application performing the computation also needs some preparation to be able run on the client machines under the control of the Core Client. Apart from having executables for all possible platforms that are member of the DG, the application also has to be prepared to be run by the BOINC Core Client which has two main aspects:

  • it should be able to run in the directory structure used by the client, i.e. application executables are placed in the project directory while the working directory is a separate slot directory where input and output files are linked and
  • it should be able to interact with the Core Client, i.e. handle suspend, resume and quit requests and report used CPU time and checkpoints.

BOINC provides an API that applications should use to communicate with the Core Client and handle running in the DG environment. This API provides functions for resolving links to files that are accessed from the slot directory, communicate exit status to the Core Client so it can handle errors and report statistics as needed.

Legacy applications or applications which cannot be modified to use the API are not able to run under BOINC because without calling the right API functions they would find links instead of their input files, write their outputs to the wrong place and without properly reporting statistics to the Core Client the application would be restarted over and over and eventually it would be marked as failed. For these applications BOINC offers the BOINC Wrapper (see figure above) which acts as a main program managing communication with the Core Client calling the appropriate API functions and running the real application executable as a subprocess. An application using the BOINC Wrapper contains the wrapper executable besides the application files.

Applications with GenWrapper

A typical GenWrapper application consists of a zip file holding all the files belonging to legacy application(s), the two GenWrapper components: GitBox and Launcher executables and a profile script to perform platform specific preparations. A typical work unit for a GenWrapper wrapped application contains the input files and another shell script (the work unit shell script), which allows to control and execute the legacy applications in an arbitrary manner for each work unit. The work unit shell script should be platform independent as the work unit can be executed by any supported platform.

GitBox is a stripped-down Windows only port of BusyBox originally created for the Windows version of the git version control system which internally relies on running shell scripts. Although GIT on Windows later abandoned GitBox, this port was used as the basis for GenWrapper after extracting it from GIT and porting back to UNIX preserving functionality on Windows. Later the GenWrapper GitBox version was updated to match newer BusyBox releases, and thus diverged from the original GitBox significantly, although it is still referred to as GitBox in the GenWrapper distribution for historical reasons. Besides Windows the GenWrapper GitBox (which we will simply call GitBox now) is also supported on Linux and Mac OS X, some of the stripped down parts were put back and new BusyBox functionality (e.g., lzma compression) were added and it was extended with BOINC specific shell commands.

A GenWrapper wrapper legacy application is executed as follows. The client downloads the Launcher executable (named like the application as BOINC expects), an application zip file and an optional profile script as the BOINC application and a work unit (input files and a work unit shell script). The Launcher is started by BOINC and acts as a BOINC application, handling all communication with the Core Client. After starting, the Launcher looks for a .zip file with the same name as itself and extracts all files from it to the slot directory. Storing application files in a .zip file is optional, if not found no extracting is performed and the work unit script should access it by resolving its logical name as any other input files. It is recommended to use application zip files, because the BOINC server stores all application files in one common location on the project web server and when files with the same name, but different content are required by different applications a conflict may happen which is prevented by storing application specific files in a zip archive. The most obvious scenario would be that different applications require different versions of the same DLL (Windows shared library) files. These files commonly have the same 8 character filename (plus the ”.dll” extension) regardless of their version. Without packaging application files together in a zip, a later deployed application could overwrite the same named files belonging to a previously installed application. The application zip file may also contain a profile script which serves as a platform specific bootstrap script for the application (e.g., on Linux the library include path may need to be adjusted or local optimization options could be enabled depending on the presence of optional features, etc.). After unzipping the application archive, the Launcher generates a starter script which first sources the profile script if exists and then executes the work unit shell script. Then Launcher calls the built in POSIX shell interpreter (ash) of GitBox which starts to execute this generated script.

The Launcher remains running while GitBox executes the script and handles communication with the Core Client and performs similar tasks as the BOINC Wrapper. In fact Launcher was originallty based on BOINC Wrapper, but it is heavily modified to fit the needs of GenWrapper. Modifications include:

  • suspending and resuming GitBox and all the subprocesses started by it when the Core Client asks for this;
  • measuring and reporting the CPU time used by the running subprocesses and
  • killing the subprocesses if the requested or the client is stopped.

The Launcher is spawning a new process for GitBox which is also spawning a new process for each legacy application it is executing. If the functionality of the original BOINC Wrapper were used here, only the GitBox process could be controlled and measured, while loosing control over the legacy application processes (which do the actual work) and the Core Client having no information about them. These tasks are implemented differently on UNIX and Windows systems due to the lack of common API concepts and Windows' limited support of POSIX.

On Linux and Mac OS X there are process groups that the Launcher utilizes by simply putting the spawned GitBox process in a new process group and by default all its child processes will also belong to the same group. The limitation here is that no child process should break away from the process group, thus currently no subshells are supported (scripts should not create background processes or in practice ampersand and parenthesizes should be avoided) and also the legacy applications should avoid create new process groups (although they can spawn subprocesses which are not breaking away from the process group).

On Windows, there are no process groups (a feature with the same name exists, but it is only vaguely similar to UNIX process groups and cannot be used the same way). The closest feature the WIN32 API provides is called JobObject. Each JobObject represents a collection of processes. But the problem with them is twofold:

  • by default a process started by a process in a JobObject (child) should also belong to the same JobObject as its parent, but unfortunately it depends which system function was used to create the child process (CreateProcess() is fine, but _spawn() is not always), thus not every child process may end up in the JobObject;
  • there is no official, documented API function to suspend or resume a process, only threads can be controlled; if a process has more than one thread suspending them in the wrong order might lead to a dead-lock.

The last problem is solved by using undocumented Windows NTAPI calls that can directly suspend and resume processes. The first problem was overcome by periodically checking the list of running processes whether there is a new one whose parent process belongs to the JobObject but it isn't. If such processes are found they are added to the JobObject. There is no function to suspend or resume all processes in a JobObject, but it is possible to terminate all processes of it. Thus, if suspend or resume is requested the JobObject is queried for the list of its processes and each one of those is handled one by one.

N=`boinc resolve_filename in`
OUT=`boinc resolve_filename out`
NUM=`cat ${IN}`
PERCENT_PER_ITER=$((10000 / NUM))
for i in `seq $NUM`; do
    PC=$((PERCENT_PER_ITER * i /1000))
    boinc fraction_done_percent ${PC}
    echo -e "I am ${PC}% complete. " >> ${OUT}
    sleep 1;
done   

GitBox (as well as BusyBox) has a modular structure, which allows to easily extend it with arbitrary commands by so called applets. The BOINC extension is implemented in such an applet and currently consists of the most important BOINC API calls such as:

  • resolve_filename,
  • fraction_done,
  • fraction_done_percent

A minimalist sample work unit script for demonstrating the basic capabilities can be seen above. This sample is reading an (integer) value from the file with the logical filename 'in' (by first resolving the link to the real file), performing a loop which:

  • calculates how much of the total work is done;
  • print the fraction done in percent into the file with logical name 'out' and
  • sleep for a second in each iteration.

There is no need to provide or call boinc_init() or boinc_finish() from the script itself, because it is called by the Launcher, which also takes care of forwarding the exit status to boinc_finish (thus the script can normally exit with a non-zero status to signal an error). The Launcher also measures used CPU time and reports to the BOINC Core Client automatically. The only BOINC API functionality required for this simple example is to resolve logical filenames and report the fraction done which is also the case for the majority of legacy applications.

component/genwrapper.txt · Last modified: 2013/01/18 12:31 by atisu
Trace: genwrapper
Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0