Real Time CustomRatesUpdate() Eat A Lot Of Disk Space !!!

New comment
 

I'm using CustomRatesUpdate() function. It costs up to 2-3GB of disk space on real time running interval 1 sec in 24 hours (version A) but while I copy M1 timeframe data from server then CustomRatesUpdate() from 8/2024 just in 7MB (version B).

CustomRatesReplace() also has the same effect


Version A (Causes problem):

string _CustomSymbol = "Custom"+_Symbol;

int OnInit() {
  ResetLastError();

  if(!CustomSymbolCreate(_CustomSymbol, "", _Symbol)) Print("Created "+_CustomSymbol);
  EventSetTimer(1);
  
  return(INIT_SUCCEEDED);
}

void OnDeinit(const int reason) {
  EventKillTimer();
}



void OnTick() {

}

void OnTimer() {
  MqlRates ratesArr[];
  ResetLastError();
  if(CopyRates(_Symbol, PERIOD_M1, 0, 1, ratesArr) == -1) {
    Print("CopyRates Error #"+(string)GetLastError());
  } 
  else if(CustomRatesUpdate(_CustomSymbol, ratesArr) == -1) {
    Print("CustomRatesUpdate Error #"+(string)GetLastError());
  }
}

Version A Result (in 1.5 hour):


Version B:

string _CustomSymbol              = "Custom"+_Symbol;
input datetime downloadHistoryFrom       = D'2024.08.01 00:00';

int OnInit() {
  ResetLastError();

  if(CustomSymbolCreate(_CustomSymbol, "", _Symbol)) Print("Created "+_CustomSymbol);

  ResetLastError();
  MqlRates ratesArr[];
  if(CopyRates(_Symbol, PERIOD_M1, downloadHistoryFrom, TimeCurrent(), ratesArr) == -1) {
    Print("Error on CopyRates #"+(string)GetLastError());
  } 
  CustomRatesUpdate(_CustomSymbol, ratesArr);

  return(INIT_SUCCEEDED);
}

void OnDeinit(const int reason) {

}

void OnTick() {

}

Version B Result:


I wonder if running CustomRatesUpdate() function will delete old records

 

Just a thought - for the 1-st scenario try to call CustomRatesReplace once a while for the history built with previous CustomRatesUpdate calls - probably this will pack the files.

CustomRatesUpdate is designed to update quotes "visually", and probably is not optimized for storage size.

 
Stanislav Korotky #:

Just a thought - for the 1-st scenario try to call CustomRatesReplace once a while for the history built with previous CustomRatesUpdate calls - probably this will pack the files.

CustomRatesUpdate is designed to update quotes "visually", and probably is not optimized for storage size.

I have tried the following cases:

1. CustomRatesReplace() 

2. CustomRatesUpdate()

3. CustomRatesDelete() before CustomRatesReplace()

4. CustomRatesDelete() before CustomRatesUpdate()

Everything that runs in real time take up a lot of disk space and there is no difference between them

Don't know if there is a solution to this

 


You can try to verify your created/downloaded history data:

      string _CustomSymbol = "Custom"+_Symbol;

      long tstart_server = SeriesInfoInteger(_CustomSymbol, PERIOD_M1, SERIES_SERVER_FIRSTDATE);
      long tstart_terminal = SeriesInfoInteger(_CustomSymbol, PERIOD_M1, SERIES_FIRSTDATE);
      long tlast_terminal = SeriesInfoInteger(_CustomSymbol, PERIOD_M1, SERIES_LASTBAR_DATE);

      Print("* ", _CustomSymbol);
      Print("Total number of bars for the symbol-period at this moment = ",
            SeriesInfoInteger(_CustomSymbol, PERIOD_M1, SERIES_BARS_COUNT));
      Print("The first date in the history for the symbol-period on the server = ",
            (datetime)tstart_server);
      Print("The first date for the symbol-period at this moment = ",
            (datetime)tstart_terminal);
      Print("Lastbar date for the symbol-period at this moment = ",
            (datetime)tlast_terminal);
 
Soewono Effendi #:


You can try to verify your created/downloaded history data:

Tested with version B:


Furthermore, I set max bars in chart = Unlimited. Surely I have copied enough M1 candlestick history from the input time
 
Compare to Version A ?
 
Soewono Effendi #:
Compare to Version A ?

Version A is real time CustomRatesUpdate(), Surely the bar data will be less than version B, but somehow version A takes up a lot of disk space after a while of running.

 
According to statistics, when I run version A for 1 day (24 hours) with any symbol, it consumes 3GB of storage, the history is 1 day. While I copy historical data rates from August 1, 2024 -> today (version B), it only takes up 7MB of memory. This is absurd
 
you still can run the code I provided to see the information of your custom symbol, right ?
Are the information displayed correct ? especially total number of bars ?
 
Soewono Effendi #:
you still can run the code I provided to see the information of your custom symbol, right ?
Are the information displayed correct ? especially total number of bars ?

yes 

Version A (Run In 3 Mins, run after delete history)


 
Try to run it over longer period and verify the information again.

You might want to verify file size in the custom folder:

<DATAFOLDER>Bases\<CUSTOM FOLDER>\history\<CUSTOM SYMBOL>\

and

<DATAFOLDER>Bases\<CUSTOM FOLDER>\history\<CUSTOM SYMBOL>\cache\


Notice there are .hcc custom symbol folder and its *.hc files in the cache folder.

Good luck.
12
New comment