A DC-API/ BOINC Generic Wrapper for legacy applications.

GenWrapper is a generic wrapper interpreter (scriptable as the POSIX shell). It is very much based on GitBox that was extracted from git and modified to run standalone. GitBox is a stripped down version of BusyBox modified to compile under Windows with MinGW. This version should also compile on Linux and Mac OS X (and probably more but those are not tested) besides Windows. Main features are:

• Runs legacy applications without BOINCification.
• Makes BOINC API/ DC-API available in POSIX shell scripting.
• Available for Mac OS X, MS Windows, GNU/ Linux (it is based on GitBox).
• Small size (~600k).

• Precompiled binaries can be found at the files section on SourceForge.
• The homepage of GenWrapper can be found here.
• Git repository is at GitHub: https://github.com/atisu/genwrapper.git .

GenWrapper can be compiled as a native BOINC application or as a DC-API application (see here for more information on DC-API).

GenWrapper consists of two parts: a launcher (gw_launcher{.exe}) and GitBox (gitbox{.exe}). The launcher acts as a DC-API/BOINC application, it’s task is to prepare the application and communicate with BOINC, while GitBox provides a generic DC-API/BOINC enabled scripting interface.

Any application using GenWrapper must have the following components:

• The launcher, renamed to the name of the registered application in BOINC (e.g., numsys-1.00_i686-apple-darwin)
• Either:
  • A zip file with the same name and .zip extension holding the legacy application and !GitBox binaries. GitBox on Windows must be named as gitbox.exe on Linux/ Mac OS X as gitbox)
  • Or the application binaries and GitBox executable unzipped. GitBox on Windows must be named as gitbox.exe on Linux/ Mac OS X as gitbox.
• An optional script insalled as part of the application named profile.sh (profile script). This script will be executed each time before the application is started. This is optional. The optional profile script should follow these rules:
• Should be a standard POSIX shell script (no bash or ksh extensions). It can be tested locally via GitBox (execute gitbox sh profile.sh).
• It is run after uncompressing the application archive, but before starting the work unit.
• Should handle all platform/architecture specific tasks required for the application to run (e.g., set library path).
• Exported environment variables are later available for the work unit supplied script.

Work units for any GenWrapper application must follow these rules:

• They should supply a shell script (work unit supplied script), which:
  1. Is a standard POSIX shell script (no bash or ksh extensions). It can be tested locally via GitBox (execute gitbox sh scriptname).
  2. Should contain platform/ architecture independent code only.
  3. Should execute the legacy application(s).
  4. Has access to environment variables set in the profile script.
• The first command line argument of the work unit should be the name of the work unit supplied shell script.

Available BOINC/ DC-API commands:

boinc resolve_filename <FILENAME> 
boinc fraction_done <0.00 .. 1.00> 
boinc fraction_done_percent <0 .. 100> 

The exit <STATUS> command equals to calling the boinc_finish(<STATUS>) BOINC C API function. Notes:

• gitbox, gw_launcher are BOINC compound applications.
• DC-API enabled version also uses these shell commands but instead of the BOINC API calls the appropriate DC-API functions.
• Legacy applications executed by the shell cannot contain any BOINC API / DC-API functions (especially init and finish functions).

Available non-BOINC specific shell commands:

ash, awk, basename, bunzip2, bzcat, cat, chmod, cmp, cp, cut, date, diff, 
dirname, dos2unix, echo, egrep, env, exit, expr, false, fgrep, find, grep, 
gunzip, gzip, head, ls, lzmacat, md5sum, mkdir, mv, od, patch, printenv, 
printf, pwd, realpath, rm, rmdir, sed, seq, sh, sleep, sort, sum, tac, tail, 
tar, test, touch, tr, true, uniq, unix2dos, unlzma, unzip, wc, which, yes, 
zcat, [, [[ 

The following shows a simple example for a work unit supplied script:

wu_script.sh

IN= ‘boinc resolve_filename in‘ 
OUT= ‘boinc resolve_filename out‘ 
NUM= ‘cat "${IN}"‘ 
PERCENT_PER_ITER=$((10000 / NUM)) 
for i in ‘seq ${NUM}‘ ; do 
    PERCENT_COMPLETE=$((PERCENT_INTER *i / 1000) ) 
    boinc fraction_done_percent ${PERCENT_COMPLETE} 
    echo -e "I am ${PERCENT_COMPLETE}% complete." >> "${OUT}" 
    sleep 1 ; 
done 

It reads an integer from a logical file named in and runs a loop that many times. In each iteration it sleeps for a second and writes the completion status to the logical file named out while reporting it to the BOINC client. Notes:

• Every filename needs to be resolved via boinc resolve_filename.
• Variable references should be properly quoted (e.g., NUM=‘cat “${IN}“‘).
• Only the launcher is DC-API/ BOINC enabled, GitBox and shell scripts act as legacy applications.
• exit <STATUS> shell command equals to calling the boinc_finish(<STATUS>) BOINC C API function.
• Applications executed by the script cannot contain any BOINC/ DC-API functions, must be legacy applications.

A GenWrapper wrapped application works as follows:

1. A BOINC client downloads an application and a work unit.
2. The main executable of the application is the launcher for GenWrapper. This launcher is executed by the client.
3. The launcher is a DC-API/ BOINC enabled application, thus it contacts the client after start.
4. The launcher checks if a zip file named <application_name>.zip exists, if yes it unzips it.
5. The launcher generates a shell script which:
   • Checks if a file named profile.sh exists, if yes it sources it.
   • Executes GitBox with the command line arguments of the work unit, where the first argument must be the name of the work unit supplied shell script. The launcher will resolve the name of this script, it is enough to provide the logical file name.
6. The generated shell script is executed.
7. When the script finishes the exit status of the work unit supplied script is set as the exit status of the work unit.
8. The launcher exits and the results are uploaded to the server.

More legacy executables may be bundled together and run, for example one executable could use the output of another executable in a single work unit, even if there are special preparations to be done with the output before using it as input.

GenWrapper provides log files when requested:

• Standard error (stderr.txt) and
• Standard output (stdout.txt)

These files should be specified as output files e.g., with wsclient (see code below) and will be automatically generated by GenWrapper.

<WRAP box center>

 wsclient \
 [...]
 -o stderr.txt \
 -o stdout.txt

<WRAP centeralign>A GenWrapper log example with wsclient</WRAP> </WRAP>

GenWrapper can be compiled as standalone, BOINC enabled or DC-API enabled. It’s been tested and working on Debian GNU/Linux, MS Windows XP and Mac OS X 10.5. For compiling on Windows use the MINGW toolkit, MS Visual Studio is not supported. It uses a single Makefile, the compile options must be set there.

Configurable options in the Makefile:

# setting DCAPI to "yes" will set BOINC to "yes" also 
BOINC=yes 
DCAPI=yes 
ifeq ($(findstring mingw, $(TARGET)), mingw) 
DCAPI_HOME=C:/Projects/dcapi/trunk 
BOINC_CFLAGS=-IC:/Projects/boinc_mingw/include -IC:/Projects/openssl/include 
BOINC_LIBS=-LC:/Projects/boinc_mingw/ -lboinc -LC:/Projects/openssl/lib \
      -lcrypto -Wl,-Bstatic -lstdc++ -Wl,-Bdynamic -lm 
#LAUNCHER_CFLAGS= 
#LAUNCHER_LDFLAGS= 
#only when compiling with mingw and BOINC is set to "yes" 
#OPENSSL_DIR=C:/Projects/openssl/lib/ 
else 
DCAPI_CFLAGS= ‘pkg-config --cflags dcapi-boinc-client‘ 
DCAPI_LIBS= ‘pkg-config --libs dcapi-boinc-client‘ 
BOINC_CFLAGS=-I/usr/include/BOINC 
BOINC_LIBS=-lboinc_api -lboinc -lcrypto -Wl,-Bstatic -lstdc++ \
       -Wl,- Bdynamic -lpthread -lm 
endif 

• The BOINC=yes|no and DCAPI=yes|no options (above) decide whether DC-API or BOINC support should be compiled. On Linux and Mac OS X no other adjustment is needed.
• On Windows the DCAPI_HOME, BOINC_CFLAGS and BOINC_LIBS variables should be set to the directories where the BOINC/ DC-API libraries and includes reside.
• Linking libraries compiled with MinGW and MS Visual C++ can be troublesome, we recommend to compile the BOINC and DC-API libs via MinGW too.

For cross-compiling you should set the TARGET variable. (Additionally setting CC, AR and STRIP or other variables may also be necessary if defaults are not sufficient).

To set up cross-compiling environment:

1. Get the ’Cross-Hosted MinGW Build Tool’ script from the MinGW download page (Tested with x86-mingw32-build.sh-0.0-20061107-1).
2. Set the package versions in x86-mingw32-build.sh.conf, tested with the versions shown below.
3. Run the script to get and build cross-compiling environment
4. To compile simply do:

   make TARGET=/path/to/mingw/install/dir/bin/i386-mingw32 

To compile with the Cygwin compiler install the mingw versions from setup.exe and then you can issue the following command:

make TARGET=mingw CC="gcc -mno-cygwin" AR=ar STRIP=strip 

First make sure that the log files are retrieved. Usually they provide enough information to find out the cause of the problems. Refer to Log files section.

Use the BOINC client command line version. When you start it with –help (i.e., “boinc –help”) it will print out a lot of useful command line parameters. The two important ones are the following:

• --exit_before_start
• --exit_after_finish

You should have a test BOINC project set up. Make sure that only the client you are going to start next is connected, and there is no unsent work units in the project. Then proceed as follows:

1. Submit a single work unit for your application to the
   manual/genwrapper.txt · Last modified: 2014/12/29 16:53 (external edit)