I. Specifications

The Messaging Service Specification page provides a high level description of the intended OOI CI messaging service, based on AMQP broker technology.

II. Models

AMQP 1.0 Models

AMQP 1.0 removes the Exchange concept of 0.8/0.9 and provides a more abstract way of specifying multiple types of MOM entities. In AMQP1.0 Broker and Client are just two kinds of Applications related with the classical view of MOM. This specification is more generic, and allows for having producers/consumers into brokers; also, apps can also act as brokers for proxying, etc.
The following models provide an overview of the changes in this version.

 
 
AMQP 1.0 Frame Model (simplified)

 
AMQP 1.0 Session Model

Applications may define their units of work (e.g., work on 1MB pieces), whereas network connections may define smaller pieces (e.g., 64k frames). This is particularly important when dealing with very large messages and enables opportunistic computing - the session doesn't necessary have to assemble the entire message before passing it to an upper layer. Processing could also be done on parts of the message.

AMQP 1.0 Session Endpoint State Model (simplified)

AMQP 1.0 Session Endpoint State Model Decomposed

AMQP 1.0 Link Model

AMQP 1.0 Connection Model

Channels are local representations of sessions in the connection endpoint space.
Incoming "session" routing tables relate sessions with channels in the "local" connection-endpoint address space.

III. Implementation Deliverables

Here, the design and implementation of the Messaging Service Layer is presented as an artifact delivered at the conclusion of the CI Development Iteration 2 (Iteration Zinc). The context of the document reflects the working understanding and working terminology used in code during the iteration.

Source tree at end of revision

Messaging Service Layer with Magnet

Magnet introduced a simple framework for a basic application messaging layer over the AMQP messaging layer.

Core Classes in Magnet Source Code

• Agent (Pole)
• Role
• ChannelManagmentLayer (Adapter between Pole and AMQP Client)
• AMQClient (txAMQP implementation/interface to AMQP Connection and Channel Class)

The Magnet class Agent is derived (conceptually) from magnet.pole.Pole, the word polewill be used throughout the rest of the document. (This was done to prevent breaking other projects being developed with Magnet. The next phase of development will clean up the class names.)

• Poles are entities in the Messaging Service.
• A pole is instantiated with a resource_name and a unique_id
• A pole consumes messages from it's own AMQP queue. The queue is bound to an exchange with two routing_keys:
  o the generic resource name (within the exchange)
  o a unique name
• A pole contains a collection of roles.
• A role contain a collection of actions.
• An action encapsulates the ability to do something. (the end point of a message)

• ChannelManagmentLayer connects the pole to an txAMQP channel.

The entities in the iteration 2 prototype were ambiguous in differentiating Services from Agents (in OOI design terms).
A pole was supposed to be like an Agent, but it could just as easily be used to to construct any service interfaced via the Messaging Service.

Addressing

Addressable services/resources:

• Controller
• Provisioner
• Scheduler
• etc.

A message is sent between to poles like this:

pole.sendMessage(ResourceName, RoleName, RoleMessage)

ResourceName is an address within the messaging service. It's corresponding AMQP exchange_name and routing_key are looked up by the pole. In the next iteration, this should develop into the Messaging Service Registry.

Message Format

AMQP Message:

• routing_key (Pole Resource Name, or Pole Unique Id)
• Payload (Pole Message)

Pole Message:

• Role Name
• Role Message

Role Message:

• Action Name
• Action args

Analysis and Assessment of Python AMQP Client Libraries

• txAMQP - Asynchronous library for twisted
• Qpid - Asynchronous? library implemented using threading
• py-amqplib Synchronous library

txAMQP is the Qpid library adapted to work in Twisted.
txAMQP/Qpid generates AMQP classes from the AMQP spec dynamically using Python MetaClasses.
txAMQP/Qpid is built around non-human readable AMQP classes dynamically generated from the AMQP XML Spec at every runtime.

amqplib is built around human readable AMQP class source code generated one time from the AMQP XML Spec.

How the clients work

There are 3 distinct jobs an AMQP client must fulfill:

• Send/Receive TCP data
• Send/Receive AMQP Frames
• Send/Receive AMQP Methods

• TCP Layer (Socket): binary data stream
  o (Consume) TCP read binary data, process into AMQP frame
  o (Produce) TCP de-multiplex, send frames to channel/consumer receiver
• Frame layer: frame stream
  o (Consume) AMQP de-code frame into AMQP method
  o (Produce) AMQP send to method handler
• AMQP Method Layer method stream

amqplib does a fine job modularizing these distinct jobs.

txAMQP is not as modular. The file/class names don't always reflect purpose/function.