DC-API

1 Introduction

The DC-API was created by MTA SZTAKI to allow easy implementation and deployment of distributed applications on multiple grid environments. In order to accommodate the needs of very different grid environments, the DC-API supports only a restricted master-worker programming model. The restrictions include:

  • Master-worker concept: there is a designated master process running somewhere on the grid infrastructure. The master process can submit worker processes called work units.
  • Every work unit is a sequential application.
  • There is support for limited messaging between the master and the running work units. It can be used to send status and control messages, but it is not suitable for parallel programming.
  • There can not be any direct communication between work units.

2 General concepts

2.1 Programming model

DC-API applications consist of two major components: a master application and one or more client applications. The master is responsible for dividing the global input data into smaller chunks and distributing these chunks in the form of work units. Interpreting the output generated by the work units and combining them to form a global output is also the job of the master.

The master application usually runs as a daemon, but it is also possible to write a master that runs periodically (e.g. from cron), processes the outstanding events, and exits.

Client applications are simple sequential programs that take their input from the master, perform some computation on it and produce some output.

Writing a master application

A typical master application does the following steps:

  • Initializes the DC-API library by calling DC_initMaster() function.
  • Calls the DC_setResultCb() function and optionally some of the DC_setSubresultCb(), DC_setMessageCb(), DC_setSuspendCb() and DC_setValidateCb() functions, depending on the features (messaging, subresults etc.) it wants to use.
  • In its main loop, the master calls the DC_createWU() function to create new work units when needed. If the total number of work units is small (depending on the grid infrastructure), then the master may also create all the work units in advance. If the total number of work units is too large for this, the master may use the DC_getWUNumber() function to determine the number of running work units, and create new work units only if this number falls below a certain threshold.
  • Also in its main loop the master calls the DC_processMasterEvents() function that checks for outstanding events and invokes the appropriate callbacks.
  • Alternatively, the master may use the DC_waitMasterEvent() and DC_waitWUEvent() functions instead of DC_processMasterEvents() if it prefers to receive event structures instead of using callbacks.

Writing a client application

A typical client application performs the following steps:

  • Initializes the DC-API library by calling DC_initClient() function.
  • Identifies the location of its input/output files by calling the DC_resolveFileName() function.

The client application may not assume that it can read/create/write any files other than the names returned by DC_resolveFileName().

  • During the computation, the client should periodically call the DC_checkClientEvent() function and process the received events.
  • If possible, the client should call the DC_fractionDone() function with the fraction of the work completed. On some grid infrastructures (e.g. BOINC) this will allow the client's supervisor process to show the progression of the application to the user.
  • Ideally the value passed to the DC_fractionDone() function should be proportional to the time elapsed so far compared to the total time that will be needed to complete the computation.
  • The client should call the DC_finishClient() function at the end of the computation. As a result all output files will be sent to the master and the master will be notified about the completion of the work unit.

2.2 Messaging

The DC-API provides limited messaging functionality between the master application and the clients. The DC-API has the following features and restrictions:

  • Messages are not reliable in the sense that if the client is not actually running when a message is being sent to it (e.g. because it is queued by the backend grid infrastructure), then the message may be silently dropped.
  • The ordering of messages is not neccessarily maintained.
  • Messages are delivered asynchronously. There is no limit for the time elapsed before a message is actually delivered.

Due to the above restrictions, DC-API messages are not suitable for message-based parallel processing. They are meant for sending short status messages about long-running operations, or for sending control messages like a command to cancel a given computation.

3 Configuration

The DC-API library uses a configuration file to control its operations. The location of the config file must be specified by the master application when it calls the DC_initMaster() function.

Individual work units may also use a config file that has a slightly different format than the master's config file. Work unit config files however are usually generated by the DC-API library on the master side so applications should not be concerned about it.

3.1 Configuration file format

The DC-API configuration file contains key-value pairs organized into groups. The full syntax description can be found in the Desktop Entry Specification, but we also give a brief explanation below.

Lines starting with # are considered comments and are ignored. Groups are started by a line containing the group's name between brackets ([ and ]). Groups are terminated by the start of an other group or by the end of the file.

A group may contain a list of key = value assignments, each in a separate line. White space arount the = character is ignored. Note that the DC-API do not use the localized string feature of the Desktop Entry Specification.

Configuration values can be of 4 types:

  • Simple strings. According to the format specification, they must be UTF-8 encoded.
  • Integer values.
  • Floating point values. For both integer and floating point values, the following suffixes are recognized:
  kb, kib	A multiplier of 2^10
  mb, mib	A multiplier of 2^20
  gb, gib	A multiplier of 2^30
  min	        A multiplier of 60
  h, hour	A multiplier of 3600
  day	        A multiplier of 86400
  • Boolean values. The strings true, yes and on means true, the strings false, no and off are considered false.

3.2 Generic configuration items

The DC-API configuration file should contain a group called [Master] that defines global configuration options. There may be also groups called [Client-CLIENTNAME] where CLIENTNAME is the logical name of a client application the master wishes to start (the same string that is passed to the DC_createWU() function).

Default values for client-specific configuration options may also be specified in the [Master] group. If the same key is also present in a client-specific group, the client-specific value will override the value in the [Master] group.

Configuration values for the master application

The following entries can be specified in the [Master] group. Grid backends may require and/or allow further items; refer to the backend-specific documentation.

Parameter Explanation
WorkingDirectory REQUIRED. The working directory of the master application. The value must be an absolute path. The DC-API sets the master's current directory to the value specified here when the DC_initMaster() function is called.
LogLevel OPTIONAL. Specifies the verbosity of log messages generated by the DC-API or by the master application if it uses the DC_log() or DC_vlog() functions. The allowed values in the order of increasing priority are Debug, Info, Notice, Warning, Error and Critical, the default being Notice.
LogFile OPTIONAL. Specifies the name of the file where log messages should be written to. If not specified, log messages are sent to the application's standard output.

Configuration values individual work units

Parameter Explanation
LogLevel OPTIONAL. Specifies the verbosity of log messages generated by the DC-API or by the client application if it uses the DC_log() or DC_vlog() functions. The allowed values in the order of increasing priority are Debug, Info, Notice, Warning, Error and Critical, the default being Notice.
LogFile OPTIONAL. Specifies the name of the file where log messages should be written to. The value must be a relative file name without any path components. The default value is determined by the DC-API when it creates the work unit and generally it should not be overridden.
SendCfgKeys OPTIONAL. Specifies a ;-separated list of configuration keys that should be sent to the client application when a work unit is created. LogLevel and LogFile are always included so those should not be listed. The client application may use the DC_getCfgStr(), DC_getCfgInt(), and DC_getCfgBool() functions to access the values of the keys specified by SendCfgKeys
DefaultPriority OPTIONAL. Specifies the default priority of the work unit. The priority is an integer number; its allowed range and exact meaning depends on the grid backend. The default value is 0.

Further BOINC related configuration options are detailed in Section 4.2.

4 BOINC specific information

4.1 Terminology

The term workunit means approximately the same thing for DC-API as for BOINC.

Both the DC-API and BOINC uses the term result, but they mean different things. In BOINC, results are instances of a work unit waiting to be downloaded or currently under execution. The DC-API result is what BOINC calls the canonical result. This means that when BOINC generates multiple results (e.g. for redundant computation), the DC-API will not be notified about the status of individual BOINC results; instead, it will be notified only if the canonical result is found or the whole work unit is marked as failed by BOINC.

In the following sections, result will mean the DC-API term, while BOINC result will refer to the BOINC definition.

The DC-API master application is an assimilator in BOINC terms.

4.2 Configuration options

Master side

Important note: the directories specified by WorkingDirectory, ProjectRootDir and the upload & download directories specified in BOINC's config.xml must all reside on the same filesystem since the DC-API uses the link() and rename() system calls.

Parameter Description
InstanceUUID REQUIRED. The value must be a Universally Unique Identifier. The value must be unique for every master application running on the same grid backend. If two master applications are started with the same InstanceUUID value, their behaviour is undefined.
ProjectRootDir OPTIONAL. The location of the project's root directory. This is the directory that contains config.xml and other BOINC-related subdirectories.
UploadURL OPTIONAL. The upload handler's URL to send output files of processed BOINC workunits to.
InputURLRewriteRegExpMatch OPTIONAL. This variable along with the InputURLRewriteRegExpReplace varaible can be used to rewrite input file URLs based on regular expressions. The variable InputURLRewriteRegExpMatch defines the match part of the regular expression, whereas the varibale InputURLRewriteRegExpReplace defines the replacement part of the regular expression. An example value of this variable is
attic://([^/]*).*/([^/]*)$.
InputURLRewriteRegExpReplace OPTIONAL. This variable along with the InputURLRewriteRegExpMatch varaible can be used to rewrite input file URLs based on regular expressions. The variable InputURLRewriteRegExpReplace defines the replace part of the regular expression, whereas the varibale InputURLRewriteRegExpMatch defines the match part of the regular expression. An example value of this variable is
http://\1/dl/redir/\2\nhttp://localhost:12345/data/\2.

Per-client configuration

Parameter Description
Redundancy OPTIONAL. Integer value specifying the quorum required to consider the work unit as valid. The default value is 1. If this value is N,
* N + log(N) initial BOINC results will be created. 
  If one of them finishes, a new one will be 
  created automatically until the work unit 
  either succeeds or fails.
* The work unit will be considered failed 
  if more than N + log(N + 2) + 1 BOINC results fail.
* The work unit will be considered failed if there are
  N + log(N + 2) + 1 successful results but the validator 
  could not find a canonical result.
* The work unit will be considered failed 
  if the state of the work unit
  is still not decided after 2 * (N + log(N + 2))
  BOINC results have been received.

When the redundancy is greater than 1, the work unit can not be suspended using DC_suspendWU(). In the following the options are listed which allow fine tuning redundancy. These options are mutually exclusive with Redundancy.

MinQuorum OPTIONAL. Integer value specifying the quorum required to consider the work unit as valid. The default value is 1.

This option is mutually exclusive with Redundancy. MinQuorum, TargetNResults, MaxErrorResults and MaxTotalResults should be used combined.

TargetNResults OPTIONAL. Integer value specifying the number of initial BOINC results to be created. The default value is MinQuorum.

This option is mutually exclusive with Redundancy. MinQuorum, TargetNResults, MaxErrorResults and MaxTotalResults should be used combined.

MaxErrorResults OPTIONAL. Integer value specifying the maximum number of failed BOINC results for a work unit. The default value is 0.

This option is mutually exclusive with Redundancy. MinQuorum, TargetNResults, MaxErrorResults and MaxTotalResults should be used combined.

MaxTotalResults OPTIONAL. Integer value specifying the total number of BOINC results for a work unit. The default value is MinQuorum.

This option is mutually exclusive with Redundancy. MinQuorum, TargetNResults, MaxErrorResults and MaxTotalResults should be used combined.

MaxSuccessResults OPTIONAL. Integer value specifying the maximum number of successful BOINC results for a work unit. The default value is MinQuorum.

This option is mutually exclusive with Redundancy. MinQuorum, TargetNResults, MaxErrorResults and MaxTotalResults should be used combined.

MaxOutputSize OPTIONAL. Max. size of any output files the client application generates. The default is 256 KiB. If the size of an output file exceeds this value, the BOINC core client will not upload that file and will report the BOINC result as failed.
MaxMemUsage OPTIONAL. Max. memory usage of the client application. The default is 128 MiB. Hosts with less available memory will not download work units for this application. Also, if the applications's real memory usage exceeds this limit, the BOINC core client aborts the application and reports the BOINC result as failed.
MaxDiskUsage
EstimatedFPOps OPTIONAL. The estimated run-time of the client application, expressed in the number of floating point operations. The default is 1013. This value is used by the BOINC server to decide whether a given host is eligible to run a work unit and is also used by the BOINC core client for scheduling decisions.
MaxFPOps OPTIONAL. Max. CPU usage of the client application, expressed in the number of floating point operations. The default is 1015. If the application uses more CPU time than this value divided by the CPU's speed, then the BOINC core client aborts the application and reports the BOINC result as failed.

As per recommendations in the BOINC documentation, the value of MaxFPOps should be several times larger than the expected run time of a work unit on an average host.

DelayBound Time in seconds the BOINC server waits for a result to finish. If a client has donwloaded a BOINC result and did not finish in the given time, the result is considered failed and a new one is generated.

If DelayBound is smaller than the estimated run time of the application on a given host (calculated by dividing EstimatedFPOps by the host's speed), then the BOINC result will not be offered for download. If no host is fast enough to complete the application within the specified time limit, the result will remain unsent for an unspecified amount of time and DC-API will receive no feedback for it.

EnableSuspend OPTIONAL. Boolean value telling if work units for this client can be suspended using DC_suspendWU() or not. The default value is false.

When the redundancy is greater than 1, the work unit can not be suspended using DC_suspendWU(), regardless of the value of this configuration option.

NativeClient OPTIONAL. Boolean value telling if the client application uses the native BOINC API instead of DC-API. This will prevent adding DC-API specific input and output files to the workunit description.

Considerations for BOINC configuration

If you want to use master-to-client messaging, you must enable it in the BOINC project's configuration by making sure that the <msg_to_host/> tag is present in config.xml. Client-to-master messaging is always enabled and does not require configuration.

4.3 BOINC backend specific issues

Deploying the application

Deploying the application consists of two steps: registering the client application(s) in the BOINC database, and running the master daemon.

All client applications should be compiled for every platform you need, and installed under the project's apps directory. The BOINC name of the client application must be the same as the master uses when it calls DC_createWU(). See the BOINC documentation about how the client binaries should be named and placed and how they should be registered in the database.

The most common method of deploying the master application is to run it as a BOINC daemon by adding it to BOINC's config.xml. See the BOINC documentation for details. Other methods of deploying the master application depending on how it was designed are also possible, but the following rules must be fulfilled:

  • The master application must have access to the BOINC project's config.xml
  • The BOINC file_deleter process must have enough privileges to be able to remove files and directories created by the master application. If the master runs under the same user account as the BOINC daemons, this is usually not a problem.
  • The master application must be able to create files and directories under the project's download directory, and it must be able to access files under the project's upload directory.

Besides the master and client applications, you must also define a validator for the application in config.xml. If you are not using redundancy then you may use the sample_trivial_validator that comes with BOINC. This validator accepts everything without checking.

If redundancy is desired, you may use the validator_for_dcapi validator which does a textual (meaning converting between UNIX and Windows line endings) comparison of the first output file.

If you are running multiple master applications under the same BOINC project, and you want to use sample_trivial_validator for any of them, then you must use it for all of them. This restricition exists for any other validator that is not DC-API aware, since it can not determine which work unit belongs to which master and therefore which results should it validate and which ones should it leave alone.

Redundant computation

Redundancy is very important if you are running computations on untrusted clients instead but may even be useful on dedicated clients to protect from hardware failures. Besides deliberate tampering with the output, clients may also produce incorrect results due to hardware problems like bad memory, overheating or faulty CPU or simply disk corruption.

Redundant computing means sending the same work unit to multiple different clients and comparing the results. The comparison is performed by a tool BOINC calls validator. The validator usually is application-specific as it must understand the output file format to filter out unimportant noise (like different line endings on different operating systems, or small differences between floating point results due to the different rounding characteristics of different CPU architectures).

Redundancy can be enabled in DC-API on a per client application basis by adding the appropriate Redundancy value to the client's configuration group.

If redundancy is enabled for a client application, work units for that client can not be suspended. The reason that it is generally impossible to compare the state of two BOINC results suspended at two different stage of their execution. If one of the suspended results is already corrupt and is restarted, the validator will no longer recognize the corruption since all new results starting from the corrupted initial state will produce the same but bad output.

Messaging

BOINC provides a limited messaging support that is accessible thru the DC-API DC_sendWUMessage() and DC_sendMessage() functions on the master and client side, respectively.

See the note about configuration requirements for master-to-client messaging.

BOINC messaging has several restrictions:

  • Messages can only be sent to BOINC results that are currently running. If a work unit has no running result, messages sent to it are silently discarded.
  • If redundancy is enabled, master-to-client messages are sent to all running BOINC results regardless their state. In case of client-to-master messages, the master cannot tell which BOINC result sent the message. This means that “request-response” style messaging is hard to implement correctly when redundancy is enabled.
  • Messages sent by the master are delivered only when the client connects to the master next time. Since the master has no control over this, the client should periodically send messages to the master to force a connection if timely receiving of messages sent by the master is important. Be caraful about the extra load placed on the BOINC server by clients sending messages too frequently.
  • When multiple messages are being queued in either direction due to the client not connecting to the server frequently enough, they will be delivered to the peer in an undefined order.

Cancelling a running work unit

The DC_cancelWU() function can be used to cancel a running work unit. This function is implemented by sending a special message to all running BOINC results. This implies that unless clients where BOINC results for this work unit are running connect back to the BOINC server, the cancel request may not be delivered until the client finishes the computation.

Due to a race condition between various components of the BOINC system, it is also possible that a new BOINC result is created and is sent out after the work unit has been cancelled. Such BOINC result will not receive the cancellation request and will run until it finishes its computation. Its result however will not be reported to the DC-API master, so the application should not be concerned about this.

When is a result reported

The BOINC core client handles the completion of a BOINC result in two phases: first it uploads the output files, then it notifies the BOINC server that the result has been finised. The validator will notice the completion of the BOINC result only when this notification is received.

However, this notification is sent only when the core client has to connect the BOINC server the next time, which may be a long time if the core client has already started processing the next BOINC result while the output files of the previous result were being uploaded.

When there are no more work units to download, the client sleeps for a couple of minutes before trying again. This means that the reporting of the completion of the last work unit may be delayed for a couple of minutes even after all its output files have been uploaded.

The DC-API master application will receive notification about a result when the validator has made its decision. This may also introduce some delay after all BOINC results have been completed.

Work unit priority

The priority of a work unit can be set either by using the DC_setWUPriority() function or by specifying it in the configuration file using the DefaultPriority key. Either way, the priority can be an arbitrary 32-bit integer.

The BOINC scheduler dispatches higher priority work units first. Results belonging to work units with lower priorities will not be offered to clients until all the higher priority work units are exhausted.

Common errors

There are some common errors:

Type of error Suggested todos
No results are reported Check the validator. When there is no validator defined in config.xml or the validator fails for some reason, the DC-API master will not receive result notifications.
The final result is not reported There is a couple minutes delay before reporting the final result. It is normal.
I've fixed a bug in a client application, but results are still computed using the old client Be sure to give the new client binary with a version number greater than the old client, or otherwise the clients will not notice that the binary has been updated and will not download it.

Open issues

The following list contains the known problems with DC-API's BOINC backend:

  • Messages are not removed from the msg_to_host and msg_from_host tables by the db_purge tool, so they need to be cleaned up manually from time to time to prevent the database from being filled up.
  • The DC-API creates result template files in the templates subdirectory in the project's root directory, but those files are never removed.

5 Compiling the Uppercase example

In this section the procedure of compiling a BOINC application (“Uppercase”) using the DC-API will be demonstrated.

5.1 Linux

You can find the test application - Uppercase-example - in the following directory: /usr/share/doc/libdcapi-common-dev/examples/uppercase-with-callback

Preparing the environment

If you haven't done so yet at the server installation section, add the following line to the /etc/apt/sources.list file:

  deb http://www.desktopgrid.hu/debian/ squeeze szdg

Run

$ apt-get update

In case you receive a PGP error, run

$ apt-key advanced --keyserver keys.niif.hu --recv-keys 6A2165907B1AAC6F

Install the DC-API development package, that contains the necessary libraries.

$ apt-get install build-essential
$ apt-get install libdcapi-boinc-dev
$ apt-get install libdcapi-doc
$ apt-get install make
$ apt-get install autoconf
$ apt-get install libtool

The commands above will also install some other packages that are needed for compiling. You can compile applications under your own user account, no special privileges are required for application compiling.

Copy the source

The test application - Uppercase-example which we are going to compile - is in the following directory:

/usr/share/doc/libdcapi-common-dev/examples/uppercase-with-callback

Create a directory named 'uppercase' in your home directory and copy the contents of /usr/share/doc/libdcapi-common-dev/examples/uppercase-with-callback into it.

Compiling

Uncompress the source files, execute autoconf, configure and finally make to get the sources compiled.

$ gzip -d *.gz 
$ autoreconf -if
$ ./configure
$ make

After the compilation finishes, you can find the executable of the master and client application in the same directory. Later for the deployment you will need the following files:

  • uppercase-example-master - binary of the master application
  • uppercase-example-client - binary of the client application
  • uppercase-example-master.xml - config file of the master for the deployment step
  • uppercase-example-client.xml - config file of the client for the deployment step

The source of the application is prepared to be easy understandable and modified. After you've become familiar with compiling the applications, you can modify the functionality of them as necessary for you own purpose using this example as skeleton.

After the application is compiled, you can continue on the next page with the deployment of the application.

5.2 Linux/MinGW32: Cross-compiling for Windows

This is only for the client part of the application. See the previous section for how to compile the master part of the Uppercase application.

We provide a package that contains an example application (“Uppercase”) and precompiled DCAPI and BOINC libraries and headers for MinGW32.

  1. Please download and unzip the Uppercase example package for MinGW32
  2. Read the README file for setting up a MinGW32 environment
  3. Run make to compile the application (client.exe).

You can use this sample application (it's source code is in the src directory) as a skeleton for developing your own DC-API applications. If you cannot find your binary, check for error messages. In case there are DC-API related problems, please contact us!

If you prefer to compile DC-API on your own and not using our package, follow the instructions found in the README-MinGW32 in the DC-API source. The referred package for building the BOINC libraries for MinGW32 can be found here.

5.3 Windows using Visual Studio

In order to ease the compilation for the windows version of Uppercase example, we prepared a package. This package for Windows® contains static libraries that supports application development for the desktopgrid architecture on client side. Inside this package is a complete build environment for Microsoft® Visual C++® 2008 (Express Edition which is free) together with the Uppercase example application, that demonstrates the basic structure of an application utilizing DC-API.

  1. Please download and unzip the Uppercase example package for windows
  2. Open the following file in the package using your MSVC++ 2008: uppercase-example-client-win\win_build\uppercase-example-client-win.sln
  3. Rebuild your application with MSVC++ 2008

Now, you must have the uppercase-example-client-win.exe binary in the uppercase-example-client-win\win_build\Debug\ directory. If you cannot find your binary, check for error messages in your compilation window and fix them if any. In case there are DC-API related problems, please contact us!

For a detailed reference manual for DC-API please, visit this page.

6 Deploying applications on a BOINC server

  • Keep in mind to maintain your server as a server administrator user and not via the root or a simple user account.
  • Also keep in mind to always stop your project before maintenance, by executing the stop command. After you have finished maintenance restart the server with the start command. All the effects of your maintenance will take effect when you restart the server.
  • Keep in mind that all of the project maintenance procedures have to be carried out with a user administrator having boinc project admin rights. Do not forget to “sudo” to the boinc admin user account, which also has the home folder set to /var/lib/boinc/<project short name>. Most of the files and folders mentioned in this section are located on this path, in the subdirectory named project.

After you have prepared your application to be executed in the BOINC environment, you have to deploy the application on the server. Deploying an application is an easy task and has to be carried out only once for each application.

6.1 Deploying the client part of the Uppercase example on BOINC

Do not forget that the home folder of the project is /var/lib/boinc/<project's short name>/project and to access this folder with a user account having project administrator rights. The steps of the install process is as follows:

Step1: Add the application to the project.xml file of your project.

<app>
  <name>uppercase-example</name>
  <user_friendly_name>Example Application</user_friendly_name>
</app>

The name tag will identify your application in the project, while the users will see the user friendly name. It is very important that you give the exact same name here what you named the folder in /<project name>/project/apps/

Step2: Copy the executable and all of the necessary files of the application in the apps folder of your project. The naming convention of the application executables and directories in BOINC is: <name>_<version #>_<platform>. There are several pre-defined platforms in BOINC. For example the Linux Intel® x86 platform is identified by the i686-pc-linux-gnu string and the Windows Intel® x86 platform is identified by windows_intelx86. The version numbering should start from 1.00 and increment in 0.01 steps for each new version of the application.

Please note, that in case of a windows application, the directory name has to match the name of the application exactly, also containing any extensions, like '.exe'.

Follow the steps to create a Linux binary for Linux based clients.

  cd ~/project/apps/
  mkdir uppercase-example
  cd uppercase-example
  mkdir uppercase-example_1.00_i686-pc-linux-gnu.exe
  cp <path of compilation>/uppercase-example-client uppercase-example_1.00_i686-pc-linux-gnu/uppercase-example_1.00_i686-pc-linux-gnu

Please note, that if you want to create a binary for Windows based clients, you will have to set up an environment for compiling applications on Windows.

Step3: The application has to be signed to ensure the integrity and security of the application. This step is optional, since the final step of executing the update_versions script can carry out the application signing procedure automatically, if it finds the signatures to be missing. Before signing the executables ensure that all of the executable files has been granted executable rights on the server, including Windows executables. If an application consists of multiple files (e.g.: like windows DLLs), then all of files has to be signed separately.

  sign_executable uppercase-example_1.00_i686-pc-linux-gnu ../../../keys/code_sign_private >uppercase-example_1.00_i686-pc-linux-gnu.sig

The key files of your project are located in the ~/project/keys folder. If you prepare to run a public desktopgrid, you should keep your private keys in a secure place (e.g.: on a removable media, like a CD).

Step4: Install a validator for your application. The job of the validator is to ensure that the results returning from the clients are intact and has not been altered in a bad way on the client. This is an application dependent task, so the creator of the application has to write the validator as well, which compares returning results and decide whether they are acceptable or not. If you run your desktopgrid in a local environment and does not use redundancy than it is not necessary to validate the results. However, a validator is still needed. For this an example validator can be used which is provided in the server package. If you use your own validator copy the executable in the bin directory of your project. To install the example validator add the followings to the config.xml file of your project, pay attention to the executable name if you use your own validator.

  <daemon>
    <cmd>
      sample_trivial_validator -d 3 -app uppercase-example
    </cmd>
  </daemon>

Step5: Execute the xadd command if you are installing an application for the first time. This script reads the contents of project.xml and commits any new applications or platforms found into the database of the project. Later, when updating the version of your application you do not have to run this command.

Step6: Execute the update_versions command, to commit the latest version of your application in the database of your desktopgrid. This command has to be executed, whenever you install a new version of your application.

6.2 Deploying the master part of the Uppercase example on BOINC

Please, note that you need to follow your installation from this point only if you would like to use a work generator that is running on your BOINC server undeer your project in the background as a daemon. To generate workunits, there are other alternatives, like 3G Bridge.

Step1: Log in to the server as a project administrator and go to the home directory (/var/lib/boinc/<short name of project>/) and create a directory called <application name>-master

  mkdir uppercase-example-master

Step2: Copy the file named 'dcapi.conf' (located in /usr/share/doc/libdcapi-common-dev/examples/uppercase-with-callback) in this directory and rename it to 'uppercase-example-master.conf'. Personalize this configuration file similarly as the example below. Don't forget to generate a unique uuid and use it instead of the example below. You can use the uuidgen command-line utility which can be found in the uuid-runtime package.

  WorkingDirectory = /var/lib/boinc/<project's short name>/uppercase-example-master
  
  InstanceUUID = fc7286ea-d8ab-4304-be55-84c098d7db42
  
  BoincConfigXML = /var/lib/boinc/<project's short name>/project/config.xml
  
  ProjectRootDir = /var/lib/boinc/<project's short name>/project

Step3: The input of the example application has to be located in the WorkingDirectory, which has been set in the configuration file of the above example. Create a file called input.txt in this directory and fill it with several lines of arbitrary text. The master application will create workunits by splitting this file into workunits. The master application will create 'number of lines in input.txt divided by 20' number of workunits. For example if input.txt has 20 lines the master application will create only 1 workunit, or if input.txt has 64 lines than the master application will create 4 workunits, however the 4th workunit will only has 4 lines to process, while the first 3 will have 20.

Step4: Copy the executable of the master application in the bin directory of your project.

  cp uppercase-example-master /var/lib/boinc/<project's short name>/project/bin

Step5: Run the master by executing:

  uppercase-example-master -c /var/lib/boinc/<project's short name>/uppercase-example-master/uppercase-example-master.conf

You can also put the master executable to the bin directory of the project and create a new <daemon> tag for it in the config.xml file. The <daemon> tag has to be nested in the <daemons> tag within the config.xml file. This way the master application will start and stop with the server.

  <daemon>
    <cmd>
      uppercase-example-master -c /var/lib/boinc/<project's short name>/uppercase-example-master/uppercase-example-master.conf
    </cmd>
  </daemon>

Please note that the Uppercase example application is already in the project.xml file if you use SZTAKI-BOINC package. There is no need to add it again.

If by any chance the daemon would not start then try to give the full path to uppercase-example-master executable in the bin directory.

From now on, the start script of the BOINC server will execute the example master application as well, which will create workunits based on the settings carried out above. After the example master application has created the workunits, it will wait for any results available and process them upon the BOINC server reports that they have been processed and are available for assimilating.

To enable the periodically running tasks change the 1 in the <disable> tags to 0 for each task.

Now that you have installed a BOINC server and deployed an application, you have to connect some clients to the server, which can start processing the workunits.

7 API Reference

  • Common — functionality available on both the client and master side.
  • Client — functionality specific to the client-side of DC-API.
  • Master — functionality specific to the master-side of DC-API.
manual/dc-api.txt · Last modified: 2013/07/16 07:51 by smith
Trace: dc-api
Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0