resourse management in iCustom

New comment
 

I've written code that captures the signals of an indicator for different inputs and at various timeframes.

In version 4, this was straightforward, and obtaining the indicator's output was possible with a single command.

in MQL4 I also draw a table for 10 symbols and 8 TF and 5 indicator for each of them!!


However, in version 5, it has become a bit problematic!

We first need to create a handle.

Now, my question is whether this should only be done during initialization in the OnInit function?

Or can I create the indicator handle for different inputs in the middle of the program for each timeframe, get the signal, release the resources, and then move on to the next timeframe?


I have 5 indicator handle for eatch TF , and sweep for about 10 TF in MQL5, 

to time-slice the calculation 

I sweep 1 TF in a function and every 1 second call that function with a different TFindex , that called from OnTimer()


I donot Know why CopyBuffer takes too long time  in debug!  almost  (50s)  !!

 
Mehrdad Sarrafi: Now, my question is whether this should only be done during initialization in the OnInit function?

Yes, that is the better way to do it.

Mehrdad Sarrafi: Or can I create the indicator handle for different inputs in the middle of the program for each timeframe, get the signal, release the resources, and then move on to the next timeframe?

You can also do it that way, but it will be slower and more resource intensive. When you instantiate the indicator, it needs to prepare all the data first, and then you lose all the initial preparation once you release it.

So, if you are going to continuously loop over different time-frames, you are just repeating the initialisation over and over again. That is inefficient. Instead initialise it only once, even if it is during the OnTick() event handling. One way is to use classes, using globally scoped objects of those classes for each instance, where the construct instantiates the indicator, and the destructor releases it.

If you have the source code for the indicators, then other possible methods include, rewriting the indicators to be more efficient (using multi-time-frame incremental methods), or calculating their values in the EA itself.

 
Fernando Carreiro #:

 Instead initialise it only once, even if it is during the OnTick() event handling. One way is to use classes, using globally scoped objects of those classes for each instance, where the construct instantiates the indicator, and the destructor releases it.

If you have the source code for the indicators, then other possible methods include, rewriting the indicators to be more efficient (using multi-time-frame incremental methods), or calculating their values in the EA itself.

thanks alot Fernando 

then it is better to release only at the end.

Yes I have written a class and only overwrite the vertual check function for my different project.

then I can create handles in check function that is called in OnTimer.  and then release them only on destructor .


Can I have e.g. 300 indicator handles in my expert or my parent indicator that call other indicators and combine the logical conditions ? ( for my 2TF  or 2-3combinational alerts)

and if yes , you think it is better to create all of  them in a function like myclass.init_handles() that uses a for   and creat all of them in OnInit()

or create  them in myclass.checkfun() and call it in OnTimer() event , or it doesnot differ ?

I mean does iCustom needs cpu and memory resourses that it face problem when we create 300 or more handles in OnInit or not ?


do we have a maximum number of handles that can be created in an experts or indicators?

 

Mehrdad Sarrafi #: Can I have e.g. 300 indicator handles in my expert or my parent indicator that call other indicators and combine the logical conditions ? ( for my 2TF  or 2-3combinational alerts)

... do we have a maximum number of handles that can be created in an experts or indicators?

I don't know what limit there might be on the number of indicator handles, but for 300 which then also call other indicators, I would seriously rethink the entire logic.

300 iCustom() handles for indicators that themselves access other indicators, would be a serious drain on resources and slow down your back-testing tremendously.

Personally, for that many dependencies, I would rewrite the indicator logic into the EA, while refactoring all of that logic to make it more efficient. I would also redo all the calculations to be incremental in nature as much possible to reduce the RAM and CPU consumption.

 
Fernando Carreiro #:

If you have the source code for the indicators, then other possible methods include, rewriting the indicators to do more efficient (using multi-time-frame incremental methods), or calculating their values in the EA itself.

Yes I have written the indicators codes by myself 

but in my sweep class I only check each indicator when a new candle arrives in each TF ,

do you think  using multi-time-frame incremental methods can help to be more efficient ? or combining the conditions for different indicators in differnt TF becomes more difficult?

it is kind of you if you post the link of some good project or articles that you have read in MQL site

Regards

 
Fernando Carreiro #:

I don't know what limit there might be on the number of indicator handles, but for 300 which then also call other indicators, I would seriously rethink the entire logic.

300 iCustom() handles for indicators that themselves access other indicators, would be a serious drain on resources and slow down your back-testing tremendously.

Personally, for that many dependencies, I would rewrite the indicator logic into the EA, while refactoring all of that logic to make it more efficient. I would also redo all the calculations to be incremental in nature as much possible to reduce the RAM and CPU consumption.

thanks ,

You know , logic is not so complicated !

but I only want some indicators in differnet TF to receive alarm with combinational condition 

or screening for different symbols

and I donot want to backtest it

I back test it only with 2 TF not 10 TF 

and only with 1 symbol not 10 symbols , and back test is not my concern in this project 

 
Mehrdad Sarrafi #: do you think  using multi-time-frame incremental methods can help to be more efficient ? or combining the conditions for different indicators in differnt TF becomes more difficult?

Yes, most definitely. I don't know exactly what you doing, but from my own experience using incremental methods, it greatly improves the efficiency of any indicator or EA, for both CPU and RAM usage.

Mehrdad Sarrafi #: it is kind of you if you post the link of some good project or articles that you have read in MQL site

There is probably no such article as far as I know. Incremental methods are about math and code logic and they are advanced concepts.

An example would be how one can calculate the EMA incrementally ...

You can also calculate SMA incrementally as well, and many other indicators can also be adapted in a similar fashion.

You can also, calculate values incrementally for multiple time-frames, while processing only the lower time-frame, instead of processing multiple time-frames independently.

 
Fernando Carreiro #:

Yes, most definitely. I don't know exactly what you doing, but from my own experience using incremental methods, it greatly improves the efficiency of any indicator or EA, for both CPU and RAM usage.

There is probably no such article as far as I know. Incremental methods are about math and code logic and they are advanced concepts.

An example would be how one can calculate the EMA incrementally ...

Thanks alot


I think MT5 developers consider Incremental calculation  in coding of EMA  and so iAO , iMACD

What is EMA? How to Use Exponential Moving Average With Formula
What is EMA? How to Use Exponential Moving Average With Formula
  • www.investopedia.com
An exponential moving average (EMA) is a type of moving average (MA) that places a greater weight and significance on the most recent data points. The exponential moving average is also referred to as the exponentially weighted moving average. An exponentially weighted moving average reacts more significantly to recent price changes than a...
  
ENUM_TIMEFRAMES TF_HD[10] =
  {
   PERIOD_M5,
   PERIOD_M15,
   PERIOD_M30, 

   PERIOD_H1,
   PERIOD_H2,
   PERIOD_H4,
   PERIOD_H12,
   PERIOD_D1, 
   PERIOD_W1,
   PERIOD_MN1
  };
int tfi = 0;
int h1[10];
int copied_n;
double ind1_vb1[20];
//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
//--- create timer
   EventSetTimer(1);
//---
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
//--- destroy timer
   EventKillTimer();
  }
//+------------------------------------------------------------------+
//| Expert tick function                                             |
//+------------------------------------------------------------------+
void OnTick()
  {
//---

  }
//+------------------------------------------------------------------+
//| Timer function                                                   |
//+------------------------------------------------------------------+
void OnTimer()
  {
   ResetLastError();
   h1[tfi]=iAO(NULL, TF_HD[tfi]);
   copied_n = CopyBuffer(h1[tfi], 0, 0, 20, ind1_vb1);
   if(copied_n <= 0)
      {
      Print(GetLastError());
      return;
	}
//---
   printf(EnumToString(TF_HD[tfi]));
   tfi++;
   if(tfi == 10)
     {
      tfi = 0;
     }
  }

I do my test with a simple indicator  like AO

surprisingly the handle  is created but CopyBuffer take about 48s and doesnot copy anything and the program print nothing in experts tab!!!

I have 2 question

1: why it takes almost 48s in debug mode to reach from the breakpoint of line CopyBuffer to breakpoint of next line?!!!

2: why it does not copy from buffer


the result is the same when I create the handles in OnInit() function !!

the error is 4806 ( requested data not found)

and it does not differ if array passed to copybuffer define dynamic like below

double ind1_vb1[]

  
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
  {
//--- destroy timer
   EventKillTimer();
   for(int i = 0; i < 10; i++)
     {
      if(h1[i] != INVALID_HANDLE)
        {
         IndicatorRelease(h1[i]);
        }
     }
  }

after adding this code and restarting MT5 

problem solved 


I wonder if deleting class like belowe , will call the deconstructor or not .

I am going to test it and maybe  releasing  the indicators at the end will  solve my problem with my main program!

myclasname *sw;

//+------------------------------------------------------------------+
//| Custom indicator initialization function                         |
//+------------------------------------------------------------------+
int OnInit()
  {
   sw = new Tr2TF_HD;
}

void OnDeinit(const int reason)
  {
   delete sw;
  }
New comment