Scripts: TickCompressor

New comment
 

TickCompressor:

Convert MqlTick-s into minified structures to free RAM or store tick arrays in a compact files.

TickCompressor

Author: Stanislav Korotky

 

Didn't come across situations where a flag would be needed. Therefore, it seems superfluous.

If you compress MqlTick to a useful one, then it may be worth looking towards increments. Their meanings are painfully small. Including millisecond time.


The virtual testing code you have given shows well that it can be carried out in sequential chunks. Therefore, you can fit into a very small amount of memory even if you need a backtest on a sequence of billions of ticks.

 
fxsaber:

The virtual testing code you have given shows well that it can be carried out in sequential chunks. Therefore, you can fit into a very small amount of memory even if you need a backtest on a sequence of billions of ticks.

I'm not sure where can I get the chunks if not in a large master-array of all ticks for the whole period of testing, so it would face the same problem. If you mean reading from a file by chunks, then this will also affect performance.

 

Hello,

I'm unable to compile the .mq5 file

Is there a pre-complied version ?

Cheers

 
Abdul Hakeem:

I'm unable to compile the .mq5 file

Is there a pre-complied version ?

It should compile without an issue. Provide more details about your problem.

 
Hello,.
I loaded it into the MetaEditor, then press compile. I saw several error messages. But I tried several times to no avail.
Would you by any chance have a pre-compiled version ?
Many thanks
 
Abdul Hakeem:
Hello,.
I loaded it into the MetaEditor, then press compile. I saw several error messages. But I tried several times to no avail.
Would you by any chance have a pre-compiled version ?
Many thanks

You should show the errors. Without this no one can help you.

  
void OnStart()
{
  const double Value = 128.00001;
  
  Print(NormalizeDouble(Value, 5) == NormalizeDouble((float)Value, 5)); // false
}
 
fxsaber:

Yes, this is inevitable side-effect of the compression, based on type size reduction. A citation:

Float values have between 6 and 9 digits of precision, with most float values having at least 7 significant digits. Double values have between 15 and 18 digits of precision, with most double values having at least 16 significant digits

Probably can invent a kind of correction bits, but this will make the whole solution more complicated.
 
Stanislav Korotky:

Probably can invent a kind of correction bits, but this will make the whole solution more complicated.

struct MqlTick_Short
{
private:
  uint bid;
  uint ask;
    
  struct TIMEMSC
  {
  private:    
    uchar Bytes[5];    
  
    union TIMEMSC_UNION
    {
      uchar Bytes[5];
      long Value;
    };
      
  public:
    void operator =( const long time )
    {
      TIMEMSC_UNION Union = {0};
  
      Union.Value = time - D'2000.01.01' * 1000;
      
      ::ArrayCopy(this.Bytes, Union.Bytes);
    }
    
    long operator []( const int = 0 ) const
    {
      TIMEMSC_UNION Union = {0};
      
      ::ArrayCopy(Union.Bytes, this.Bytes);
      
      return(Union.Value + D'2000.01.01' * 1000);
    }
  } time_msc;
  
  //MaxPrice = 536870911 / MathPow(10, digits)
  //digits = 6, MaxPrice = 536.870911.
  //digits = 5, MaxPrice = 5368.70911.
  //digits = 4, MaxPrice = 53687.0911.
  uint PriceToInteger( const double Price, const uint digits = 5 ) const
  {
    return(((uint)(Price * ::MathPow(10, digits) + 0.1)) | (digits << (sizeof(uint) * 8 - 3)));
  }
  
  double IntegerToPrice( const uint Price ) const
  {
    const uint digits = Price >> (sizeof(uint) * 8 - 3);
    
    return(::NormalizeDouble((Price & (UINT_MAX >> 3)) / ::MathPow(10, digits), digits));
  }  
  
public:
  void operator =( const MqlTick &Tick )
  {
    this.bid = this.PriceToInteger(Tick.bid);
    this.ask = this.PriceToInteger(Tick.ask);
    
    this.time_msc = Tick.time_msc;
    
    return;
  }
  
  MqlTick operator []( const int = 0 ) const
  {
    MqlTick Tick = {0};
    
    Tick.bid = this.IntegerToPrice(this.bid);
    Tick.ask = this.IntegerToPrice(this.ask);
    
    Tick.time_msc = this.time_msc[];
    Tick.time = (datetime)Tick.time_msc / 1000;
    
    return(Tick);
  }
};

void OnStart()
{
  MqlTick Tick1[1];
  
  if (SymbolInfoTick(_Symbol, Tick1[0]))
  {
    const MqlTick_Short TickShort = Tick1[0];
    
    MqlTick Tick2[1];
    Tick2[0] = TickShort[];
    
    ArrayPrint(Tick1);
    ArrayPrint(Tick2);
  }
  
  Print(sizeof(MqlTick_Short)); // 13
}
 
Can it be used for MQLRates too?
12
New comment
Reason: