User Tools

Site Tools


Sidebar

Overview

Navigation

Scenarios

Components

Manuals

  • Basic components
  • Application support
  • Additional components
    • manual:3gbplugin-cloud

      3G Bridge Cloud plug-in

      Allows using Amazon EC2/ Eucalyptus/ OpenStack/ OpenNebula resources via the EC2 SOAP/ REST interface.

      Usage

      The EC2 plug-in simply allows to start instances through 3G Bridge by submitting jobs to its queue. The job will remain running as long the cloud instance is running. To shutdown the instance, simply cancel the job.

      You need some middleware installed on a VM image (e.g., BOINC or Condor) to pool the resources. This document does not deal with the assembly of such an image, it simply assumes it is available. There are assembled images ready to be used deployed on Amazon, read more in Contextualization.

      Requirements

      • 3G Bridge (obviously)
      • EC2 API Tools 1.3 30349 or Euca2ools
        • Java (e.g., OpenJDK 1.6.0_0) for EC2 API TOOLS
      • Amazon EC2 account with credentials downloaded OR
      • Eucalyptus account with credentials downloaded
      • A Virtual Appliance (VA) deployed at Amazon EC2 or at an Eucalyptus/ OpenStack/ OpenNebula installation with e.g., Condor worker or BOINC Client installed.

      Usage

      This document uses the ec2-api-tools, if you want to access an OpenStack/ OpenNebula cloud, set 'euca2ools' as ec2-tool-type.

      This document assumes you have a working 3G Bridge installation. For instructions on how to deploy 3G Bridge refer to the 3G Bridge manual.

      Configuring 3G Bridge

      1. Configure 3G Bridge by creating a queue for the EC2 plugin in the cg_algqueue table:
        INSERT INTO cg_algqueue (grid, alg, batchsize, statistics) VALUES \
        		("Cloud", "cloud", 1, NULL);
      2. Create grid in the 3g-bridge.conf for the plug-in:
        [Cloud]
        handler = EC2
        user-data-file = /var/www/userdata.zip
        ec2-tool-type = ec2-api-tools
        region = eu-west-1
        ec2-certificate = /foo/bar/.ec2/cert-WLEN637UCDGGKHTERUJYEN7EE33WMOX3.pem
        ec2-private-key = /foo/bar/.ec2/pk-WLEN637UCDGGKHTERUJYEN7EE33WMOX3.pem
        ec2-home = /foo/bar/apps/ec2/ec2-api-tools-1.3-30349
        java-home = /usr/
      3. Add the following to the [Cloud] section of 3g-bridge.conf if you are not using Amazon EC2:
        ec2-service-url = <YOUR_EUCALYPTUS_SERVICE_ENDPOINT>
      4. Start 3G Bridge, you should see something similar:
        2010-10-18 13:54:17 INFO: Loaded plugin EC2_handler
        2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'region' is 'eu-west-1'
        2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'java-home' is '/usr/'
        2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'ec2-home' is '/foo/bar/apps/ec2/ec2-api-tools-1.3-30349'
        2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'ec2-private-key' is '/foo/bar/.ec2/pk-WLEN637UCDGGKHTERUJYEN7EE33WMOX3.pem'
        2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'ec2-certificate' is '/foo/bar/.ec2/cert-WLEN637UCDGGKHTERUJYEN7EE33WMOX3.pem'
        2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): 'ec2-service-url' is not set for Cloud
        2010-10-18 13:54:17 DEBUG: 10EC2Handler::parseConfig(): user specified data is (with '-f' / '-d' prefix) ' -f /var/www/userdata.zip'
        2010-10-18 13:54:17 DEBUG: Initialized grid Cloud using EC2
        2010-10-18 13:54:17 DEBUG: Reading algorithm data
        2010-10-18 13:54:17 DEBUG: Grid Cloud: added algorithm cloud
        2010-10-18 13:54:17 [Notice] The subresult callback is not set, subresults will not be reported
        2010-10-18 13:54:17 [Notice] The message callback is not set, messages will not be reported

      Configuring Amazon EC2/ Eucalyptus

      1. You need to create a security group ('boinc') for the region you are planning to use, where all started instances will belong to:
        ec2-add-group --region eu-west-1 boinc -d "My BOINC workers on Amazon"
      2. Depending on the middleware, you'll need to open specific port ranges for the security group. E.g., for the Condor image described in the next section:
        ec2-authorize --region eu-west-1 boinc -P tcp -p 40000-41000 -s <CONDOR_CENTRAL_MANAGER>/32 
        ec2-authorize --region eu-west-1 boinc -P udp -p 40000-41000 -s <CONDOR_CENTRAL_MANAGER>/32
        ec2-authorize --region eu-west-1 boinc -P udp -p 9618 -s <CONDOR_CENTRAL_MANAGER>/32 
        ec2-authorize --region eu-west-1 boinc -P tcp -p 9618 -s <CONDOR_CENTRAL_MANAGER>/32
      manual/3gbplugin-cloud.txt · Last modified: 2013/01/18 09:54 by atisu