Optimization slow when using custom indicator

[Pasi Hakamaki]

Hi,

Is it typical that optimization gets very slow when using a custom made indicator? I made a simple indicator and use it in my EA and the optimization is soo slow :( I also made EA that uses two simple moving averages and optimizing it is much faster. My custom indicator is not complex and I've done some profiling and figured out that it should have no bottle necks. I've done the same with my EA but still so slow optimization.

Br, Candles

[Jinsong Zhang]

i think the problem is from your indicator

[phi nuts]

song_song:
i think the problem is from your indicator

... and so,  Candles, re-write the code.

For example, some indicators are using Copy function. For the first tick, it is OK to copy the whole bars on chart, but for the next tick, just copy the most current data. 

[Pasi Hakamaki]

phi.nuts:

... and so,  Candles, re-write the code.

For example, some indicators are using Copy function. For the first tick, it is OK to copy the whole bars on chart, but for the next tick, just copy the most current data. 

Yes, I was thinking about that last night. My indicator copies rates from the whole chart i.e. all the bars every time OnCalculate() is called. That has got to be the reason. Will fix it NOW.

Br, Candles

[Pasi Hakamaki]

Did not fix it right away, but now getting there still have some problem see below:

   if(rates_total>prev_calculated)
     {
      //do calculations for whole chart again here :(
     }

Now the optimization is fast as it should, great :)  The below code should be even faster as it only does calculations for the current new bar but still strangely this did not work:

   if(rates_total>prev_calculated)
     {
      for(int i=prev_calculated;i<rates_total;i++)
        {
         //do calculations for current bar only, here :)
        }
      }

So now the expert still repeats the calculation for the whole chart when a new bar comes because I must use this:

if(rates_total>prev_calculated) //if we have new bar
     {
      for(int i=1;i<rates_total;i++)
        {
         //do calculation for the whole chart again here :(
        }
      }

Hope you understand my problem here :)

Br, Candles

[phi nuts]

Maybe try this

   int i, limit;

   if(prev_calculated == 0)     
       limit = prev_calculated;      //--- first calculation
     else 
       limit = prev_calculated - 1;  //--- 

   for(i = limit;i < rates_total && !IsStopped(); i++)
      {

      }

 

[Pasi Hakamaki]

phi.nuts:

Maybe try this

 

Of course! This will naturally work :)

Thanks again Phi :)

[serge103]

This problem still exist today after 7 years.

I have tested a simple filter indicator with the iCustom indicator call function.

This unique line of code used at every tick increaded the process time of my EA (already including more than 1000 line) by 23%.

I replaced the custom indicator by an EMA, and this increase was only 5%.

I then managed to modify the same indicator code and included it in my EA for the exact same result and the increase in processing time was less than a second.

I could not see the difference. Which means less than than 1 sec out of 2:17 min in back testing of severals months. It's more than 32 times faster.

I noticed the same thing with all indicators I tried. In some cases, it increases the processing time by a factor of 10 if we try to call it at every tick.

The only way they can be used is at every 15 min or more. In some cases, this is not accurate enough.

I repeat the same question: Why are the custom indicators so slow running?

Is there a way to be faster in backtesting?

What can we do when the original code is not available?

For an EA we are only interested in the 0 index immediate value, we don't care about past values (totally useless).

How can we avoid computing these useless visual past values??

 

Thanks.

[Harish M]

serge103:

This problem still exist today after 7 years.

I have tested a simple filter indicator with the iCustom indicator call function.

This unique line of code used at every tick increaded the process time of my EA (already including more than 1000 line) by 23%.

I replaced the custom indicator by an EMA, and this increase was only 5%.

I then managed to modify the same indicator code and included it in my EA for the exact same result and the increase in processing time was less than a second.

I could not see the difference. Which means less than than 1 sec out of 2:17 min in back testing of severals months. It's more than 32 times faster.

I noticed the same thing with all indicators I tried. In some cases, it increases the processing time by a factor of 10 if we try to call it at every tick.

The only way they can be used is at every 15 min or more. In some cases, this is not accurate enough.

I repeat the same question: Why are the custom indicators so slow running?

Is there a way to be faster in backtesting?

What can we do when the original code is not available?

For an EA we are only interested in the 0 index immediate value, we don't care about past values (totally useless).

How can we avoid computing these useless visual past values??

 

Thanks.

Observing the same issue with iCustom for the indicator I wrote. I have history data for 20 years. That is 500,000+ candles for 15 minute chart.

So, the trick is to limit the number of candles to consider for every calculation instead of all 500,000+ past price actions for every tick.

Use another variable to limit counting for number of past candles with X_history variable

To get the smooth result for your custom indicator, if it takes only 500 past candles, set it accordingly.

e.g.,

X_history = 500;

limit = Bars - count_already_calclulated - 1;

if (limit > X_history) {

      limit = X_history;

}

iMA if used for 5 period average, might be setting the X_history = 6, so it is faster.