very slow optimization after 4 or 5 passes

New comment
 

Hello all.  I'm currently optimizing my EA.  After 5 passes, I have 5 optimization results.  However, the next passes are running extremely slow.  I have tried restarting it multiple times, but I can't move forward with optimization.  If it is because of my code, then I don't know why the first few passes run quickly.  There is nothing in the journal that would suggest anything.  The only section of my code that is different than other EA's I have optimized before successfully is this:  (Note- bid_array and time_array are declared as global variables)

//--- allocate memory in the arrays for new elements  
   ArrayResize(bid_array,ArraySize(bid_array)+1);
   ArrayResize(time_array,ArraySize(time_array)+1);
//--- add the new elements to their proper arrays   
   bid_array[ArraySize(bid_array)-1]=SymbolInfoDouble(_Symbol,SYMBOL_BID);
   time_array[ArraySize(time_array)-1]=TimeCurrent(); 
//--- find the index in the array at which tick data is too old  
   int too_old_index=0;     
   for(int i=ArraySize(time_array)-1; i>=0; i--)
     {
      datetime seconds_elapsed=time_array[ArraySize(time_array)-1]-time_array[i];
      MqlDateTime time_struct;
      TimeToStruct(seconds_elapsed,time_struct);
      if (time_struct.sec>MaxSeconds)
        {
         too_old_index=i+1;
         break;
        }
     }
//--- use aforementioned index to delete tick data that is too old     
   double temp_bid[];
   datetime temp_time[];   
   ArrayCopy(temp_bid,bid_array,0,too_old_index,WHOLE_ARRAY);
   ArrayCopy(temp_time,time_array,0,too_old_index,WHOLE_ARRAY);
   ArrayResize(time_array,ArraySize(temp_time));
   ArrayResize(bid_array,ArraySize(temp_bid));
   ArrayCopy(bid_array,temp_bid);
   ArrayCopy(time_array,temp_time);
//--- exit before trading if something has gone wrong gathering data
   if(ArraySize(bid_array)!= ArraySize(time_array)) return;

 Perhaps it has something to do with "deleting" tick data that is too old by using ArrayCopy()?  And this is hazardous to the memory?  I am not sure.

 

I would appreciate any help solving this problem, let me know if there is any more information from my terminal that would be helpful.

 
oneilljo:

Hello all.  I'm currently optimizing my EA.  After 5 passes, I have 5 optimization results.  However, the next passes are running extremely slow.  I have tried restarting it multiple times, but I can't move forward with optimization.  If it is because of my code, then I don't know why the first few passes run quickly.  There is nothing in the journal that would suggest anything.  The only section of my code that is different than other EA's I have optimized before successfully is this:  (Note- bid_array and time_array are declared as global variables)

 Perhaps it has something to do with "deleting" tick data that is too old by using ArrayCopy()?  And this is hazardous to the memory?  I am not sure.

 

I would appreciate any help solving this problem, let me know if there is any more information from my terminal that would be helpful.

Can you post a screenshot of your settings for Strategy Tester please ?

EDIT: Where is this code executed ? in OnTick() ?

 
angevoyageur:

Can you post a screenshot of your settings for Strategy Tester please ?

EDIT: Where is this code executed ? in OnTick() ?

Yes, this code is executed in OnTick().  Here is a screenshot of the Settings Tab in the strategy tester.  Also, I tried optimizing the MACD sample from metaquotes, it runs just fine.

 

 

 

 

There is the journal.  Where it says "stopped by user" is when I stopped the running optimization of the MACD sample.  

Right now, the agents are processing tasks at about 0.1%/half hour.  This is much slower than the first 4 tasks. 

 

 
oneilljo:

Yes, this code is executed in OnTick().  Here is a screenshot of the Settings Tab in the strategy tester.  Also, I tried optimizing the MACD sample from metaquotes, it runs just fine.

 

 

That comes probably from your code, but I don't know why only after 5 passes.

Anyway as your code is executed on each tick, seems to me that makes a lot of work. Maybe you can start a profiling (in MetaEditor) to see how your code is ran and eventually detect bottleneck. What is the actual size of your arrays ?

 
angevoyageur:

That comes probably from your code, but I don't know why only after 5 passes.

Anyway as your code is executed on each tick, seems to me that makes a lot of work. Maybe you can start a profiling (in MetaEditor) to see how your code is ran and eventually detect bottleneck. What is the actual size of your arrays ?

The size of the arrays are usually only about 10-30 elements.  I ran this chunk of code in a data_tester EA, printing the size of the arrays and the latest elements each time to make sure it functioned correctly.  It seemed to run fine.  Huh.  I guess I'll learn about profiling.  Please let me know if you have any other suggestions.
 
oneilljo:
The size of the arrays are usually only about 10-30 elements.  I ran this chunk of code in a data_tester EA, printing the size of the arrays and the latest elements each time to make sure it functioned correctly.  It seemed to run fine.  Huh.  I guess I'll learn about profiling.  Please let me know if you have any other suggestions.

As your problem arises after 5 passes, may be the problem comes from your optimization parameters. How can they influence your arrays, or others portions of your code ?

What is the mean time for the 5 first passes ?

 
angevoyageur:

As your problem arises after 5 passes, may be the problem comes from your optimization parameters. How can they influence your arrays, or others portions of your code ?

What is the mean time for the 5 first passes ?

This was exactly the issue, angevoyageur, thank you for putting me on the right path.   The issue came from this section of code:

      datetime seconds_elapsed=time_array[ArraySize(time_array)-1]-time_array[i];
      MqlDateTime time_struct;
      TimeToStruct(seconds_elapsed,time_struct);
      if (time_struct.sec>MaxSeconds)

 

 I forgot that time_struct.sec would not take into account if the seconds were more than 60.  

 

Anyways, thank you for your great help. 

 
oneilljo:

This was exactly the issue, angevoyageur, thank you for putting me on the right path.   The issue came from this section of code:

 

 I forgot that time_struct.sec would not take into account if the seconds were more than 60.  

 

Anyways, thank you for your great help. 

Glad it works. You are welcome.
New comment
Reason: