Object Data Storage
-------------------

A core feature of the CIYAM platform is an Object DB named ODS (Object Data Storage) which is currently being
used for Global Meta entities (e.g. Global Archives), persistent "system" variable tuples and for application
specific meta-data and information that is not very well suited to a SQL DB (such as peer node files).

It is envisioned that the ODS will be used for other useful things down the track as being in-process and not
being a SQL DB it can be a much more efficient method of storing and retrieving information.

Implementing Peristence
-----------------------

The ODS system is a far more "low level" approach to data storage compared to something like a "SQL DB" which
uses a functional language (SQL) in order to abstract the storage and query operations whereas ODS focuses on
just making object streaming as simple as possible.

The design for implementing class persistence is very non-intrusive allowing a non-persistent class to become
persistent without having to make modifications to the class itself (other than adding some "friend" function
declarations if required). This is achieved by declaring a "storable" template class and the following is how
you can even make a simple POD ("Plain Old Data") struct persistent.

Consider the POD struct "sample":

struct sammple
{
   int32_t value1;
   int32_t value2;
};

In order to support persistence for "sample" the first thing we do is create a "storable" template class:

typedef storable< sample, 0, storable_base > storable_sample;

The type of the first "storable" template type is the class we are wanting to make persistent while the third
template type will always be "storable_base". The second template type is an integer which is used to specify
an optional value to round up the storage size to be allocated for the object. For example if the value 50 is
used then an object whose size is 40 bytes would be padded with an extra 10 bytes (set to 0) and an object of
60 bytes would be rounded up to 100 bytes. This reduces the need for object data to have to be "moved" to the
end of the data storage file too often (the ODS implementation always stores object data contiguously so once
a second object is stored immediately following the first, then the first will have to be moved to the end of
the DB if it grows any bigger than its reserved size).

After creating the "storable" template there are only 3 functions that need to be implemented for persistence
and they are as follows:

int_t size_of( const sample& s )
{
   return size_determiner( &s.value1 ) + size_determiner( &s.value2 );
}

read_stream& operator >>( read_stream& rs, sample& s )
{
   rs >> s.value1 >> s.value2;
   return rs;
}

write_stream& operator <<( write_stream& ws, const sample& s )
{
   ws << s.value1 << s.value2;
   return ws;
}

The first function is required in order to determine the size of the object (for storing only) then streaming
operators for "read_stream" and "write_stream" are required in order to stream the actual data. The streaming
operations (unlike typical "iostream" ones) are binary so ODS DB files will be platform specific (in order to
make ODS platform independent its implementation would need be reworked in order to use the same "endian" for
every platform as would the "read_stream" and "write_stream" implementations).

The "storable" template is found in "ods.h" whilst "size_determiner", "read_stream" and "write_stream" are in
"read_write_stream.h". The "size_determiner" templates can determine sizes for various standard library types
such as a "pair" as well as standard containers. Likewise "read_stream" and "write_stream" support "pair" and
standard containers to make coding persistence very simple.

To see how the "size_determiner" templates as well as "read_stream" and "write_stream" operators help to make
persistence simple consider the following more complex example (which is a cut down version of "outline_base"
from "test_ods.cpp"):

class outline_base : public storable_base
{
   public:
   friend int_t size_of( const outline_base& o );

   friend read_stream& operator >>( read_stream& rs, outline_base& o );
   friend write_stream& operator <<( write_stream& ws, const outline_base& o );

   private:
   string description;
   oid_pointer< storable_file > o_file;

   vector< oid > children;
};

int_t size_of( const outline_base& o )
{
   int size_holder = sizeof( size_t );

   return sizeof( string::size_type ) + o.description.length( )
    + sizeof( oid ) + size_holder + ( o.children.size( ) * sizeof( oid ) );
}

read_stream& operator >>( read_stream& rs, outline_base& o )
{
   o.children.erase( o.children.begin( ), o.children.end( ) );

   rs >> o.description;
   rs >> o.o_file;

   rs >> o.children;

   return rs;
}

write_stream& operator <<( write_stream& ws, const outline_base& o )
{
   ws << o.description;
   ws << o.o_file;

   ws << o.children;

   return ws;
}

In this example the types of the member variables being "streamed" includes a string, an "oid_pointer" (which
is another storable object itself) and a vector of object identities (aka OIDs).

Cache Implementation
--------------------

Although actually a separate component the CIYAM cache implementation (cache.h) has been used by ODS in order
to operate as efficiently as possible. There are four caches that are used by any one ODS instance. One cache
is for the ODS index file, one for the ODS data file and one each for transaction operations and data.

In order to prevent an ODS DB from being cached by the OS file caching system specific sizes has been set for
each of the four caches. Relevant constants in ods.cpp are:

const int c_data_bytes_per_item = c_ods_page_size;
...
const int c_index_items_per_item = 128;

and in ods.h:

const int_t c_trans_ops_per_item = 64;
const int_t c_trans_bytes_per_item = c_ods_page_size;

The "page" size being used here is 4K and an index entry takes 32 bytes (therefore c_index_items_per_item has
been assigned 128). A transaction op entry requires 64 bytes thus c_trans_ops_per_item has been set to 64. If
your OS page size is bigger than 4096 (say 8194) then these sizes should be doubled otherwise the I/O will be
only half as efficient (if your page size is smaller than 4096 it shouldn't matter).

As the cache implementation uses an "unsigned int" (which can be 32 bit even on 64 bit platforms) the maximum
objects currently storable is limited to approx 548 billion with a total data size of 16TB. Although it might
be worthwhile changing the cache implementation to use a 64 bit integer down the track these limits currently
are not expected to be an issue.