Remote Procedure Call Protocol

General view

This is a simple Remote Procedure Call protocol which offers the following capabilities: - the server can implement remote methods and group them into services. - the service is actually a class. Instantiated on server start-up. If you wish to dynamically start/stop/initialize a service somehow, define custom "Start/Stop/Initialize" methods and invoke them as you would do with any other method. - the client is able to invoke server remote methods on the defined services. - custom types can be defined and used as remote call parameters. The same types must be available to both client & server. - the server must answer to every invoke with a response. Either a success response, or an error response.

Asynchronous calls

The server is free to execute the calls in parallel and should send back every result as soon as the call finishes execution. The results may not arrive back to client in the same order as the invokes were sent. If the client needs some actions in sync, it should wait for every single result before issuing the next invoke. Every RPC packet has a transaction id. The server will respond with a result having the same transaction id.

The transport

The rpc protocol does not specify a certain transport. It can be implemented using both stream transport (TCP, SSL) or datagram transport (UDP, HTTP). Once the transport connection is established, the first communication is the handshake.

The handshake

It's a 3 way handshake. Every handshake packet has a header of 5 bytes with the following format:

  • 3 bytes: the ASCII chars "rpc"
  • 1 byte: rpc-protocol-version HI byte
  • 1 byte: rpc-protocol-version LO byte

The involved packets are as follows:

  1. client -> server
    • 3 bytes: "rpc"
    • 2 bytes: rpc-protocol-version
    • 32 bytes: client random generated data.
  2. server -> client
    • 3 bytes: "rpc"
    • 2 bytes: rpc-protocol-version. Should match the client version. Otherwise don't send this packet and drop the handshake.
    • 32 bytes: server random generated data. Different from the client data.
    • 32 bytes: repeat client data.
  3. client -> server
    • 3 bytes: "rpc"
    • 2 bytes: rpc-protocol-version. Should match the server version. Otherwise don't send this packet and drop the handshake.
    • 32 bytes: repeat server data.

Every party has the chance to drop the handshake on the following cases:

  • malformed handshake packet;
  • different rpc protocol version; the party may chose to log an error msg.

If any of the parties receives a malformed handshake or the advertised protocol version differs, it should drop the connection immediately, without replying anything.

By successfully completing the handshake both parties agree on RPC version.

Example successful connect:

  1. client -> server, "rpc\0x01\0x00abcdefghijabcdefghijabcdefghijab"
  2. server -> client, "rpc\0x01\0x0001234567890123456789012345678901abcdefghijabcdefghijabcdefghijab"
  3. client -> server, "rpc\0x01\0x0001234567890123456789012345678901"

After a successful handshake all the following data comprises of RPC Packets.

Packet format

Every RpcPacket comprises of a Header and a Body. A pseudo-code description: 

RpcPacket {
 Header : {
   mark : <string: ASCII characters: "rpc">
   xid : <integer: transaction id>
   msg_type : <integer: message type: see "message type values">
   body_length: <integer: the body length in bytes>
 }
 Body : <either CallBody, or ReplyBody>
}

CallBody : {
  service : <string: the service name, plain UTF8 text>
  method : <string: the method name, plain UTF8 text>
  parameters : [
    <object: any type, different types can be mixed into this array>,
    ...
  ]
}

ReplyBody : {
  status : <integer: a code describing execution status: see "reply status values">
  result : <object: any type>
}

The message type values

RPC_CALL 0 The packet is a client query.
RPC_REPLY 1 The packet is a server response.

The reply status values

RPC_SUCCESS 0 RPC executed successfully.
RPC_SERVICE_UNAVAIL 1 No such service.
RPC_PROC_UNAVAIL 2 No such procedure in the given service.
RPC_GARBAGE_ARGS 3 Illegal number of params or wrong type for at least one parameter.
RPC_SYSTEM_ERR 4 Unpredictable errors inside server, like memory allocation failure. Errors in remote method calls should be handled and error codes should be returned as valid rpc types.
(e.g. a method like "WriteFile" can return a number signifying 0=success and (!0)=error code ).
RPC_UNAUTHORIZED 6 The user is not authorized to perform the call
TODO(cosmin): the rpc has no authorization mechanism, this status has no meaning here. Move it to application level.

The RPC Packet generic format does not specify an encoding. So how are these types: string, number, object,.. encoded? See "types and encoding".

RPC Queries

Client: sends a Call (RPC_CALL) type Packet containing:

  • service = the service you wish to invoke
  • method = the method you wish to call on the given service
  • params = {..args..} Any kind of rpc objects in here.

Server: responds with a Reply (RPC_REPLY) type Packet containing:

  1. on success:
    • status = RPC_SUCCESS
    • result = method return value, can be any type.
      If the method returns void, a void object is returned here.
      An object is certainly returned here.
  2. on failure:
    • status = failure status (one of: RPC_SERVICE_UNAVAIL, RPC_SYSTEM_ERR, ..)
    • result = either a string containing a description of the error or a simple void object for lazy servers.

RPC description language and types

To avoid duplicating the code by defining the same methods on both server and client, and to be able to define generic and custom types, independently of encoding, we created an RPC description language. Using the RPC description language we define RPC services, methods and custom types.

Types

The RPC language currently supports the following base types:

  • void: void type, useful on methods that don't have a return type.
  • bool: boolean: true or false.
  • int: Signed integer number (4 bytes): −2,147,483,648 to +2,147,483,647 .
  • bigint: Signed integer number (8 bytes): −9,223,372,036,854,775,808 to +9,223,372,036,854,775,807 .
  • float: floating point number.
  • string: holds a variable length text, UTF8 encoding.
  • array<T>: array of T objects. Where T is any type.
  • map<K,V>: map of [key, value] pairs. Where K may be any of: int, float, string. And V can be any type.
  • date: a specific datetime moment.

Custom types may be defined, as structures containing base types and/or other custom types.
Defining a custom type, example: 

Type engine {
  int power;
  optional float weight;
  string serial_number;
}
Type car {
  string name;
  engine e;
}
Type ParkingLot {
  array<car> cars;
}

Notice the OPTIONAL keyword. An OPTIONAL attribute may have an undefined value. If "undefined" the peer will also receive the attribute as "undefined". The initial motivation for this keyword: When upgrading the type to a new version, by adding new members, all the new members should be OPTIONAL. A new peer can make these members "undefined" and successfully talk to an older peer.

Service definition

You can define a service much like a C++ namespace.
e.g. 

Type Car {
  string name;
  int hp;
}
Service CarVendor {
  string VendorName();
  int CarPrice(string car_name);
  Car CreateCar(string car_name);
}

Encoding

So far there are 2 encodings specfied: Binary and Json.

Binary Encoding

All integer types are LITTLE ENDIAN encoded.

  • void:
    • 1 byte = with value: 0xff
  • bool:
    • 1 byte = with value: 1 for true, 0 for false.
  • int:
    • 4 bytes, LITTLE ENDIAN
  • float:
    • 4 bytes (IEEE 754 Standard, the most commonly encountered representation)
  • string:
    • 4 bytes = unsigned int, LITTLE ENDIAN. Contains string length in bytes.
    • "length" bytes = the actual characters. Not including the 0 end. UTF8 encoding.
  • array<T>:
    • 4 bytes = unsigned int, LITTLE ENDIAN: n = the number of items in array.
    • n x T items = every item is a complete type.
  • map<K,V>:
    • 4 bytes = unsigned int, LITTLE ENDIAN: n = the number of pairs in map.
    • n x K,V pairs = every key and value is a complete type
  • date:
    • 8 bytes = unsigned int, LITTLE ENDIAN: the number of milliseconds since January 1, 1970, 00:00:00 GMT
  • custom type:
    • string = field1 name
    • raw_bytes = field1 type value
    • string = field2 name
    • raw_bytes = field2 type value
      ...

Particular Packet Binary Encoding

HEADER:

  • 4 bytes: the ASCII characters "rpc\0x00"
  • 4 bytes: transaction id (xid) // can be anything here, as long as it's unique per connection
  • 1 byte: message type
  • 4 bytes: body length // unsigned int LITTLE ENDIAN

BODY:

The body is basically a union, containing either a CallBody or a ReplyBody depending on the message type: 

struct CallBody
{
 string service; // the service name
 string method;  // the method name
 raw_bytes parameters; // parameters passed on method call. This is a sequence of mixed types. The server, based on service & method, knows what types to expect.
}

struct ReplyBody
{
  int replyStatus;  // a code describing execution status.
  raw_bytes result; // the returned value. Can be any type, basic or custom.
}

Json Encoding

This is a text based encoding, following the JSON standards (http://www.json.org/).

  • void:
    • text "null", without quotes. e.g.: null
  • bool:
    • text "true" or "false", without quotes. e.g. false
  • int:
    • text representation, without quotes. e.g. 123456
  • float:
    • text representation, uses dot, without quotes. e.g. 3.14
  • string:
    • <quote>text<quote> . e.g. "sample text"
  • array<T>:
    • [item1, item2, ... ] . e.g. ["some", "sample", "text"]
      All items MUST be same type.
  • map<K,V>:
    • { key1 : value1, key2 : value2, ... } . e.g. { "length" : 100, "height" : 45, "weight" : 37 }
      All keys and all values MUST be same type.
  • date: the number of milliseconds since January 1, 1970, 00:00:00 GMT. e.g. 9027834509187107
  • custom type: encoded like a map<string, V> where V is any type.
    e.g. 
    Type ISBN {
  array<int> digits;
}
Type Book {
  string name;
  int pages;
  ISBN isbn;
}

A Book may be encoded as follows: 

{ "name" : "The floating opera" , "pages" : 213 , "isbn" : { "digits" : [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 0 , 1 , 2 , 3 ] } }

Particular Packet Json Encoding

Note that body_length header attribute is missing. 

For a CALL packet:
{
  header : {
    xid : <number> ,
    msgType : 0
  } ,
  cbody : {
    service : <string> ,
    method : <string> ,
    params : [<item1>, <item2>, ...]  // different item types can be mixed
  }
}

For a REPLY packet:
{
  header : {
    xid : <number> ,
    msgType : 1
  } ,
  rbody : <result_type>
}