Overview
Navigation
Scenarios
- Simple scenarios
- Complex scenarios
- Bridging scenarios
Components
- Basic components
- Application support
- Job submission interface alternatives
- Additional components
Manuals
- Basic components
- Application support
- Additional components
- manual:restif-accounting
Table of Contents
Job-level accounting information interface
Not relased yet.
[draft]
The 3G Bridge accounting interface provides information that is required by the EGI Grid Federation to accept Desktop Grids as federation components.
Schema specification
The interface provides accounting information on a job-by-job basis. The information available through this interface is described in the following table.
Name Unit Description start_time UNIX timestamp Start time of the job as reported by the DG end_time UNIX timestamp End time of the job as reported by the DG wallclock_time s Start time - End time cpu_time s Net CPU time consumed by the job memory byte Approx. memory consumed by the job (see remarks below) host_flops 1/s Floating point performance of the executing host host_intops 1/s Integer performance of the executing host host_ncpus 1 Number of processors in the executing host Remarks
- The values are extracted from the desktop grid database.
- The desktop grid executes jobs redundantly; only the canonical result is used to determine the accounting information.
- The memory consumed by the application cannot be determined exactly. The memory bound defined for the application is used as an approximation.
- This information is only available so long as the job exists in the database. After deleting the job, accounting information is lost.
Dealing with meta-jobs
The following are implementational details included for the sake of completeness; the client using this interface need not be concerned about it.
Meta-jobs present two issues.
First, the accounting information of a meta-job must be an aggregation of information on its sub-jobs. The host_flops and host_intops values are particularly problematic. For example, the host_flops of the participating hosts cannot simply be summed: each term must be weighted with the actual participation of the given host. The solution is the following. The number of floating point operations performed on each host is computed. The total number of operations divided by the total CPU time will give the actual performance of the infrastructure executing the meta-job:
aggregate_host_flops = sum(cpu_time*host_flop)/sum(cpu_time)
The aggregation function used for meta-jobs are summarized in the following table:
Name Aggregator in case of meta-jobs start_time Min end_time Max wallclock_time Max - Min cpu_time Sum memory Sum host_flops Sum(cpu_time*host_flops)/Sum(cpu_time) host_intops Sum(cpu_time*host_intops)/Sum(cpu_time) host_ncpus Sum Second, as a meta-job progresses, when a sub-job has finished, its output is processed and the sub-job is deleted immediately. Therefore, the sub-job information must be saved before it is lost. This is done in database internally, with a trigger. The saved information is aggregated to produce the accounting information of the meta-job.
Enabling the interface
The interface is provided only by the REST interface. The interface is enabled by default. To disable it, remove the corresponding line from
3gb.php
:- 3gb.php
RESTHandler::addHandler('AccinfoHandler'); ... RESTHandler::addHandler('JobsHandler');
Make sure that the registration of
AccinfoHandler
precedes the registration ofJobsHandler
, as the URL patterns of these handlers overlap.The database has to provide the necessary functionality to enable the interface. The schema is defined in
db/egi-accounting.sql
.mysql << db/egi-accounting.sql
Interface specification
The accounting information can be accessed through <job's URL>/accinfo
For example:
JOBID=e8d8e4f4-27f0-409f-ba80-f9f9e33d3242 wget "http://home.edges-grid.eu/home/download/wss/jobs/$JOBID/accinfo"
e8d8e4f4-27f0-409f-ba80-f9f9e33d3242 1361961911 1362010369 48458 1259713.9488 1339227204000 27077851019757.2 82720014025364.8 47083
The result is the record containing the job identifier followed by the fields described in the table above in order. The syntax is as described in the REST interface manual (space delimited lines by default).
manual/restif-accounting.txt · Last modified: 2013/03/11 14:17 (external edit)
Page Tools