Page Operations
Browse Space
The Nimbus context broker and agent can be used to bring up the Cloud Provisioning environment on Nimbus or EC2. It can securely deploy secrets files and exchange host information between nodes.
There is plenty of documentation about the context broker available here and here. This page just provides a specific example of bringing up a hypothetical cloud provisioning environment.
Contextualization
The context broker uses a cluster document which describes a heterogeneous cluster of VMs and the dependencies between them. It also provides arbitrary data that is injected into the VM after boot. An example cluster document for the CPE might look like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 | <cluster xmlns="http://www.globus.org/2008/06/workspace/metadata/logistics"> <workspace> <name>rabbitmq-head</name> <image>rabbitMQ.gz</image> <quantity>1</quantity> <nic wantlogin="true">public</nic> <ctx> <provides> <identity/> <role>rabbitmq-head</role> </provides> </ctx> </workspace> <workspace> <name>provisioner</name> <image>ooi-base.gz</image> <quantity>1</quantity> <nic wantlogin="true">public</nic> <ctx> <provides> <identity/> </provides> <requires> <identity/> <role name="rabbitmq-head" hostname="true" pubkey="true" /> </requires> </ctx> </workspace> <workspace> <name>scheduler</name> <image>ooi-base.gz</image> <quantity>1</quantity> <nic wantlogin="true">public</nic> <ctx> <provides> <identity/> </provides> <requires> <identity/> <role name="rabbitmq-head" hostname="true" pubkey="true" /> </requires> </ctx> </workspace> <workspace> <name>aggregator</name> <image>ooi-base.gz</image> <quantity>1</quantity> <nic wantlogin="true">public</nic> <ctx> <provides> <identity/> </provides> <requires> <identity/> <role name="rabbitmq-head" hostname="true" pubkey="true" /> </requires> </ctx> </workspace> </cluster> |
The cluster defines 4 "workspaces", the most important being rabbitmq-head, because the other three (provisioner, scheduler, aggregator) depend on it. The rabbitMQ node uses an image called rabbitMQ.gz while the others use one called ooi-base.gz (these would be AMI names on EC2). The images have a small Python program called the context agent installed and rigged into the startup scripts. At boot, each node's agent contacts the Context Broker which coordinates exchanging information between nodes. In our example, the broker would hold all VMs until rabbitmq-head was up and running. It would then provide the IP, hostname, and hostkey of rabbitmq-head to the other VMs.
The context agent handles adjusting the system through simple scripts with defined order of execution and command line parameters. In our example, each dependent VM will have a script, /opt/nimbus/ctx-scripts/1-ipandhost/rabbitmq-head. This script can be bash, python, or whatever and will be executed with the host information of rabbitmq-head: ./rabbitmq-head 10.0.3.1 foo foo.bar.com
For example, below is a script which joins other rabbitMQ nodes into a cluster with rabbitmq-head:
#!/bin/sh echo "RabbitMQ-head IP: $1" echo "RabbitMQ-head Short hostname: $2" echo "RabbitMQ-head Hostname: $3" rabbitmqctl stop_app rabbitmqctl reset rabbitmqctl cluster rabbit@$2 rabbitmqctl start_app exit 0
You can also provide key/value data in the cluster definition which will be provided to the node in the same way. So if you provide <data name="foo">bar</data>, the agent will save the value "foo" into a temp file, look for a script called /opt/nimbus/ctx-scripts/3-data/foo, and execute it with the temp file as an argument.
There are several other features which may be useful and we can always provide more context broker/agent functionality if needed.
Launching
Launching a cluster on Nimbus is easy; you just use the Nimbus cloud client and execute something like:
bin/cloud-client.sh --run --cluster my-cluster.xml --hours 1
Launching on EC2 is a little uglier, but we can improve that. Right now you have to use the cloud-client to create and monitor the context and use ec2-run-instances to actually start up the instances. However the cloud client tries to simplify this process by generating userdata files and a script of EC2 commands. So for example:
bin/cloud-client.sh --run --cluster my-cluster.xml --ec2script=/tmp/ec2script.sh
generates user data and writes a script to /tmp/ec2script.sh which looks like:
#!/bin/sh ec2-run-instances --instance-count 1 --user-data-file /opt/nimbus-cloud-client-013/history/ec2cluster-001/ctx-tmp/userdata-0 \ --kernel aki-a71cf9ce --ramdisk ari-a51cf9cc --instance-type m1.small --key default rabbitMQ.gz ec2-run-instances --instance-count 2 --user-data-file /opt/nimbus-cloud-client-013/history/ec2cluster-001/ctx-tmp/userdata-1 \ --kernel aki-a71cf9ce --ramdisk ari-a51cf9cc --instance-type m1.small --key default rabbitMQ.gz ec2-run-instances --instance-count 1 --user-data-file /opt/nimbus-cloud-client-013/history/ec2cluster-001/ctx-tmp/userdata-2 \ --kernel aki-a71cf9ce --ramdisk ari-a51cf9cc --instance-type m1.small --key default ooi-base.gz ec2-run-instances --instance-count 1 --user-data-file /opt/nimbus-cloud-client-013/history/ec2cluster-001/ctx-tmp/userdata-3 \ --kernel aki-a71cf9ce --ramdisk ari-a51cf9cc --instance-type m1.small --key default ooi-base.gz ec2-run-instances --instance-count 1 --user-data-file /opt/nimbus-cloud-client-013/history/ec2cluster-001/ctx-tmp/userdata-4 \ --kernel aki-a71cf9ce --ramdisk ari-a51cf9cc --instance-type m1.small --key default ooi-base.gz
You would then execute this script to launch the actual instances. Of course this process could be wrapped to make it a single command.
Note that the EC2 launch still relies on a Context Broker service running at the University of Chicago (or wherever).