User Tools

Site Tools


Sidebar

manual:3gbplugin-cloud

3G Bridge Cloud plug-in

Allows using Amazon EC2/ Eucalyptus/ OpenStack/ OpenNebula resources via the EC2 SOAP/ REST interface.

Usage

The EC2 plug-in simply allows to start instances through 3G Bridge by submitting jobs to its queue. The job will remain running as long the cloud instance is running. To shutdown the instance, simply cancel the job.

You need some middleware installed on a VM image (e.g., BOINC or Condor) to pool the resources. This document does not deal with the assembly of such an image, it simply assumes it is available. There are assembled images ready to be used deployed on Amazon, read more in Contextualization.

Requirements

  • 3G Bridge (obviously)
  • EC2 API Tools 1.3 30349 or Euca2ools
    • Java (e.g., OpenJDK 1.6.0_0) for EC2 API TOOLS
  • Amazon EC2 account with credentials downloaded OR
  • Eucalyptus account with credentials downloaded
  • A Virtual Appliance (VA) deployed at Amazon EC2 or at an Eucalyptus/ OpenStack/ OpenNebula installation with e.g., Condor worker or BOINC Client installed.

Usage

This document uses the ec2-api-tools, if you want to access an OpenStack/ OpenNebula cloud, set 'euca2ools' as ec2-tool-type.

This document assumes you have a working 3G Bridge installation. For instructions on how to deploy 3G Bridge refer to the 3G Bridge manual.

Configuring 3G Bridge

  1. Configure 3G Bridge by creating a queue for the EC2 plugin in the cg_algqueue table:
    INSERT INTO cg_algqueue (grid, alg, batchsize, statistics) VALUES \
    		("Cloud", "cloud", 1, NULL);
  2. Create grid in the 3g-bridge.conf for the plug-in:
    [Cloud]
    handler = EC2
    user-data-file = /var/www/userdata.zip
    ec2-tool-type = ec2-api-tools
    region = eu-west-1
    ec2-certificate = /foo/bar/.ec2/cert-WLEN637UCDGGKHTERUJYEN7EE33WMOX3.pem
    ec2-private-key = /foo/bar/.ec2/pk-WLEN637UCDGGKHTERUJYEN7EE33WMOX3.pem
    ec2-home = /foo/bar/apps/ec2/ec2-api-tools-1.3-30349
    java-home = /usr/
  3. Add the following to the [Cloud] section of 3g-bridge.conf if you are not using Amazon EC2:
    ec2-service-url = <YOUR_EUCALYPTUS_SERVICE_ENDPOINT>
  4. Start 3G Bridge, you should see something similar:
    2010-10-18 13:54:17 INFO: Loaded plugin EC2_handler
    2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'region' is 'eu-west-1'
    2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'java-home' is '/usr/'
    2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'ec2-home' is '/foo/bar/apps/ec2/ec2-api-tools-1.3-30349'
    2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'ec2-private-key' is '/foo/bar/.ec2/pk-WLEN637UCDGGKHTERUJYEN7EE33WMOX3.pem'
    2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'ec2-certificate' is '/foo/bar/.ec2/cert-WLEN637UCDGGKHTERUJYEN7EE33WMOX3.pem'
    2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'ec2-service-url' is not set for Cloud
    2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): user specified data is (with '-f' / '-d' prefix) ' -f /var/www/userdata.zip'
    2010-10-18 13:54:17 DEBUG: Initialized grid Cloud using EC2
    2010-10-18 13:54:17 DEBUG: Reading algorithm data
    2010-10-18 13:54:17 DEBUG: Grid Cloud: added algorithm cloud
    2010-10-18 13:54:17 [Notice] The subresult callback is not set, subresults will not be reported
    2010-10-18 13:54:17 [Notice] The message callback is not set, messages will not be reported

Configuring Amazon EC2/ Eucalyptus

  1. You need to create a security group ('boinc') for the region you are planning to use, where all started instances will belong to:
    ec2-add-group --region eu-west-1 boinc -d "My BOINC workers on Amazon"
  2. Depending on the middleware, you'll need to open specific port ranges for the security group. E.g., for the Condor image described in the next section:
    ec2-authorize --region eu-west-1 boinc -P tcp -p 40000-41000 -s <CONDOR_CENTRAL_MANAGER>/32 
    ec2-authorize --region eu-west-1 boinc -P udp -p 40000-41000 -s <CONDOR_CENTRAL_MANAGER>/32
    ec2-authorize --region eu-west-1 boinc -P udp -p 9618 -s <CONDOR_CENTRAL_MANAGER>/32 
    ec2-authorize --region eu-west-1 boinc -P tcp -p 9618 -s <CONDOR_CENTRAL_MANAGER>/32
manual/3gbplugin-cloud.txt · Last modified: 2013/01/18 09:54 by atisu