Distributed Objects

Author: eilemann@gmail.com
State:

• Implemented in 0.3 alpha
• Serialization using streams implemented in 0.4
• getChangeType introduced in 0.5

Equalizer provides facilities to ease the data distribution of an application. The central piece is the co::Object base class, from which distributed objects are derived. Objects become accessable by making them known to a session. The example code shipped with Equalizer contains examples of distributed objects. The Programming Guide explains distributed objects in more detail, including the easier to use variant eq::Object.

Classification

Equalizer can manage static and versioned objects.

Static objects can be instantiated on multiple nodes. Upon instantiation, the data from the master version is replicated to the 'slave' node. Static objects do not retain object data, since this data is assumed to be immutable.

Versioned objects work like a simplified version control system. One master copy of the object creates a new version whenever the application calls Object::commit. This version is pushed to all subscribers, that is, to all nodes which have mapped the object. The version data is queued on the object, and will be applied when the application calls Object::sync to synchronize a specific version, or the head version.

A simplified type of versioned objects are objects where the instance data is equal to the delta data, i.e., objects which sync their full data on each commit.

Object Mapping

To make objects distributable, they have to be known by the session. During this process the change type of object is determined.

The master instance of an object is registered using Session::registerObject. Upon registration, a session-unique identifier is assigned, which can be used to map slave instances using Session::mapObject. Mapped slave instance are instantiated with the oldest known version from the master instance, and can be synchronized to the head version using Object::sync.

Additionally, the object identifier can be used to send an ObjectPacket to another node. Each object instance has a node-unique instance identifier to address single instances of an object on a remote node.

Change Handling

During the registration of the master version of a distributed object, the way changes are to be handled is determined by calling Object::getChangeType. The change type determines the memory footprint and the contract for calling the serialization methods. The following change types are possible:

• STATIC The object is not versioned. The instance data is serialized whenever a new slave instance is mapped. No additional data is stored.
• INSTANCE The object is versioned, and the instance and delta data is identical, that is, only instance data is serialized. Previous instance data is saved to be able to map old versions.
• DELTA The object is versioned, and the delta data is typically smaller than the instance data. Previous instance data is saved to be able to map old versions.
• UNBUFFERED The object is versioned, and delta data is used to update slave instances. No data is stored, and no previous versions can be mapped. The instance data is serialized whenever a new slave instance is mapped.

Static Objects

The implementation of unversioned objects is straight-forward and requires the application to implement Object::getInstanceData.

Versioned Objects

Versioned objects override the method Object::getChangeType to indicate how changes are to be handled. They possibly have to implement, in addition to unversioned objects, the methods Object::pack and Object::unpack to create or apply a diff from the last version.

Objects with the same instance and delta data only distribute instance data for each new version, and do not have to implement pack and unpack.

Serialization

The object serialization and deserialization methods use an DataOStream and DataIStream, respectively. These streams behave like iostreams, but transfer the data in a binary format. They do no type-checking on the data, that is, it is the application's responsibility to match the order and variables during serialization and deserialization exactly. Currently they implement streaming operators for basic data types (int, float, etc.), std::strings and std::vectors of basic data types.

Object Typing

Object typing can easily be done by the application. So far no use case has emerged where Equalizer should do object typing, as described below:

  Provide type with objects:
      virtual uint32_t Object::getTypeID() const { return EQ_ID_INVALID; }
  Session::instantiateObject( const uint32_t objectID ):
      if( !get object type id from master )
          return 0
     Session::createObject( type );
     map object to objectID
     return object

Multi-Buffered Objects

Multi-buffering of objects provides each thread with a potentially different version of the object, while optimizing memory requirements when two threads happen to use the same version of an object. Below is the current design:

Requirements

  - each thread has its own version.
  - only one write thread per object across all nodes
  - sync and getVersion are thread-specific, that is, sync synchronizes this
    object to the given version for this thread only, and getVersion return the
    current version for this thread.

API

  class ???
  {
      ObjectHandle getObject( const uint32_t id, const uint32_t version );

  };
  // ObjectHandle releases object/version for reuse when it goes out of scope

Implementation

Each object has a change manager, which depends on the type of the object and its master/slave status. Equalizer implements a change manager for static objects, versioned objects with delta data and versioned objects with only instance data. Externalizing the implementation of change handling allows for optimisations in the implementation and the memory usage for storing the version data.

Open Issues

Document Session::attachObject for unmanaged objects. Document object version obsoletion.