CodeBaseSections
Watch how to download trading robots for free
Find us on Twitter!
Join our fan page
Interesting script?
So post a link to it -
let others appraise it
You liked the script? Try it in the MetaTrader 5 terminal
to pocket
Scripts

TickCompressor - script for MetaTrader 5

Stanislav Korotky
Views:
5072
Rating:
(14)
Published:
2020.08.26 13:25
TickCompressor.mq5 (1.64 KB) view
\MQL5\Include\
TickCompressor.mqh (4.61 KB) view
Download as ZIP, How to download code from MetaEditor
Need a robot or indicator based on this code? Order it on Freelance Go to Freelance

In MQL ticks are stored in a special structure - MqlTick. It's universal but a bit fat. If your algorithm handles large tick arrays it may become a reason of RAM shortage. The class TickCompressor provides a simple solution to minimize memory utilization. 

Instead of standard structure MqlTick, it offers to use 2 new data types: MqlTickBidAsk for Forex/CFDs and MqlTickBidAskLastVolume for Exchanges, as well as a bunch of method for conversion between them back and forth.

Let us compare the standard MqlTick and the mini-tick structures.

struct MqlTick
{ 
  datetime     time;
  double       bid;
  double       ask;
  double       last;
  ulong        volume;
  long         time_msc;
  uint         flags;
  double       volume_real;
};

The `time` field is excessive because it duplicates `time_msc` with lesser accuracy.

The price fields do not have to be doubles in fact. The type float is sufficient medium for most of symbols. When it comes to price calculations, then indeed double is a must, but as long as it's a matter of passive storage, float can do the job.

Also for symbols of centralized markets (not exchanges) the `last` price is always 0. In addition, `volume` and `volume_real` are also unused.

Finally, the `flags` field can easily fit into ushort.

As a result, MqlTickBidAsk for Forex and CFDs looks quite minified. 

struct MqlTickBidAsk
{
  float      bid;
  float      ask;
  long       time_msc;
  ushort     flags;
};

MqlTickBidAskLastVolume is more verbose but still compact. 

struct MqlTickBidAskLastVolume
{
  float      bid;
  float      ask;
  float      last;
  ulong      volume;
  long       time_msc;
  ushort     flags;
};

Actually, in both cases we could use `ushort spread` instead of `float ask` and build `ask` from `bid`, but it's left as an option.

The test script outputs original and minified ticks to the log.

TickCompressor test script outputs information about standard ticks and mini-ticks

For GBPUSD it uses MqlTickBidAsk, and for S&P 500 mini futures - MqlTickBidAskLastVolume.

As you can see, MqlTickBidAsk is 3 times smaller than MqlTick, and MqlTickBidAskLastVolume is 2 times smaller.

One of the areas where the tick compressor can be helpful is virtual testing and optimization using tick arrays. For example, the library Virtual provides a method for virtual trading:

static int Tester(const MqlTick &Ticks[], ...)

where the Ticks array should be provided in-memory by calling code.

If a strategy to be tested on a deep history (several years), the tick array can easily grow up to many Gbs and exceed physical RAM. In such cases one can store minified array (compressed ticks) and then patch the Virtual library to consume mini-ticks via decompression keeping all the internal logic intact (in the same way as with standard ticks).

  #ifdef TICK_COMPRESSOR
  template<typename TC>
  static int TesterTC( const TC &Ticks[], const STRATEGY Strategy, const double Balance = DBL_MIN, const bool Stop = true, const bool bNetting = false )
  {
    const int Handle = VIRTUAL::GetHandle();
    const bool Res = ::ArraySize(Ticks) && VIRTUAL::SelectByHandle(VIRTUAL::Create(Balance, (Ticks[0].time_msc / 1000 / (24 * 3600)) * (24 * 3600), bNetting));

    if (Res)
    {
      VIRTUAL::NewTickTC(Ticks, Strategy);

      if (Stop)
        VIRTUAL::Stop();
    }

    return(Res ? Handle : -1);;
  }
  
  template<typename TC>
  static void NewTickTC( const TC &Ticks[], const STRATEGY Strategy = NULL )
  {
    if (VIRTUAL::SelectOrders)
    {
      const int Size = ::ArraySize(Ticks);
      
      MqlTick result;

      if (Strategy)
      {
        for (int i = 0; (i < Size)  && (!::IsStopped()); i++)
        {
          TickCompressor::decompress(Ticks[i], result);
          VIRTUAL::SelectOrders.NewTick(result);

          Strategy();
        }
      }
      else
      {
        for (int i = 0; i < Size; i++)
        {
          TickCompressor::decompress(Ticks[i], result);
          VIRTUAL::SelectOrders.NewTick(result);
        }
      }
    }
  }
  #endif

Also the tick compressor can be used to store tick arrays in files in a more compact way.


    Galender 1.0 Galender 1.0

    This program lists and shows information about calendar events between specified date range.

    KOB Requisites Script KOB Requisites Script

    This script downloads bars and ticks from EURUSD as required for the correct execution of Kiss on Billions on EURUSD from Saeid Irani.

    Bridge - structural design pattern Bridge - structural design pattern

    Decouple an abstraction from its implementation so that the two can vary independently

    Easy Neural Network Easy Neural Network

    A native implementation of neural networks in pure MQL, bundled with an easy to use interface, with easy support of saving and loading of the network configurations after training.