CBitBuffer Class - Data Serialization in MQL5 - library for MetaTrader 5

CBitBuffer Class - Bit-Level Data Serialization in MQL5

The CBitBuffer class provides a robust foundation for bit-level data serialization in MQL5, offering fine-grained control over data storage and retrieval. The class includes support for various data types, including variable-length integers (VLQ with ZigZag encoding), and serialization of strings and structures, which are excellent for optimizing space. The class employs optimizations like internal buffering and exponential array growth to enhance performance and provides a comprehensive error handling system. It's particularly useful for network communication or file storage where minimizing data size is crucial (e.g., compression of tick data).

Key Features:

• Bit-level Operations: Allows writing and reading data bit by bit, or in specified bit lengths up to 64 bits.
• Data Type Support: Includes methods for bool, char, uchar, short, ushort, int, uint, long, ulong, datetime, float, double, string, and struct.
• Variable-Length Integers (VLQ): Implements VLQ encoding for int, uint, long, and ulong values, which can significantly save space for frequently small integer values.
• Error Handling: Provides an ENUM_BIT_BUFFER_ERROR enum and GetLastError()/GetLastErrorString() methods for robust error management.
• Buffer Management: Offers functions to clear, finalize, set raw buffer content, save to file, and load from file.
• Internal Buffering: Uses internal 64-bit buffers (m_writeBufferInternal, m_readBufferInternal) to optimize bit-level operations by accumulating bits before writing to or reading from the main ulong[] array.

//+------------------------------------------------------------------+
//| CBitBuffer Class                                                 |
//| Reads/writes individual bits or bit sequences to ulong[] buffer. |
//| Supports efficient bit manipulation and mixed read/write ops.    |
//+------------------------------------------------------------------+
class CBitBuffer
  {
public: // Core bit operations
   bool              WriteBit(bool bit);                   // Writes a single bit
   bool              WriteBits(ulong value, int numberOfBits); // Writes N bits from a ulong
   bool              ReadBit();                            // Reads a single bit
   ulong             ReadBits(int numberOfBits);           // Reads N bits as a ulong
   ulong             PeekBits(int numberOfBits);           // Reads N bits without advancing position

public: // Position and size management
   bool              SetReadPosition(long bitPosition);    // Sets read position in bits
   long              GetReadPosition();
   bool              ResetReadPosition();                  // Resets read position to 0
   bool              SkipBits(long bitsToSkip);            // Skips N bits from current read position
   long              GetTotalWrittenBits();                // Total bits written
   long              GetTotalBytesWritten();               // Total bytes written
   long              GetTotalBytesAllocated();             // Total allocated bytes
   long              GetRemainingReadBits();               // Remaining bits to read

public: // Data type specific read/write operations
   bool              WriteBool(bool value);
   bool              WriteChar(char value);
   bool              WriteUChar(uchar value);
   bool              WriteShort(short value);
   bool              WriteUShort(ushort value);
   bool              WriteInt(int value);
   bool              WriteUInt(uint value);
   bool              WriteLong(long value);
   bool              WriteULong(ulong value);
   bool              WriteDatetime(datetime value);
   bool              WriteFloat(float value);              // Writes a 32-bit float
   bool              WriteDouble(double value);            // Writes a 64-bit double
   bool              WriteString(string value);            // Writes string with length prefix
   template<typename T>
   bool              WriteStruct(T &struct_object);        // Writes struct with length prefix

   bool              ReadBool();
   char              ReadChar();
   uchar             ReadUChar();
   short             ReadShort();
   ushort            ReadUShort();
   int               ReadInt();
   uint              ReadUInt();
   long              ReadLong();
   ulong             ReadULong();
   datetime          ReadDatetime();
   float             ReadFloat();                          // Reads a 32-bit float
   double            ReadDouble();                         // Reads a 64-bit double
   string            ReadString();                         // Reads string with length prefix
   template<typename T>
   T                 ReadStruct();                         // Reads struct with length prefix

public: // Variable-length encoding for integers (VLQ)
   bool              WriteVarInt(int value);               // Writes signed int using ZigZag + VLQ
   bool              WriteVarUInt(uint value);             // Writes unsigned int using VLQ
   bool              WriteVarLong(long value);             // Writes signed long using ZigZag + VLQ
   bool              WriteVarULong(ulong value);           // Writes unsigned long using VLQ
   int               ReadVarInt();                         // Reads signed int using ZigZag + VLQ
   uint              ReadVarUInt();                        // Reads unsigned int using VLQ
   long              ReadVarLong();                        // Reads signed long using ZigZag + VLQ
   ulong             ReadVarULong();                       // Reads unsigned long using VLQ

public: // Buffer management
   void              Clear();                              // Clears buffer content and resets state
   bool              GetFinalizedBuffer(ulong &destinationArray[]); // Copies buffer content to array
   bool              SetRawBuffer(const ulong &sourceBuffer[]);     // Sets buffer content from array
   bool              Save(string filename);                // Saves buffer to file
   bool              Load(string filename);                // Loads buffer from file

public: // Error handling
   ENUM_BIT_BUFFER_ERROR GetLastError();                   // Returns last error code
   string            GetLastErrorString();                 // Returns error description string
   void              ClearLastError() ;                    // Clears last error

   void              PrintHex();                           // Prints main buffer in hex (debugging)
  };

The complete test example "CBitBuffer_Test.mq5" provided above is the best way to detect errors and verify the class's capabilities. 

Updates:

2025.07.21 - v.1.01 :

• The CBitBuffer class actively prevents illegal attempts to write after a read operation has begun, which helps in maintaining data integrity.
• Users receive a specific error (BIT_BUFFER_MIXED_OPERATION_ERROR) when they attempt to mix operations.

2025.07.22 - v.1.02 :

• Changed design of the class to allow the mixed read/write operations, by correct handling of partial flushes/refills, which helps in maintaining data integrity when switching between read and write modes.
• Removed the variable (m_operationMode) and the error code (BIT_BUFFER_MIXED_OPERATION_ERROR).
• Cleaned the code comments for better clarity of intent.
• Updated examples in "CBitBuffer_Test.mq5" to cover more test cases.