Highly inconsistent results in backtesting

 

We run backtests with "Slow complete algorithm" for a range of inputs in "Every tick based on real ticks".
In the optimization results we click a single test to run and we get totally different results.

Moreover when we use other local network agents the "Slow complete algorithm" results are different from the "Slow complete algorithm" results obtained previously through the local cores.

Does anyone have any idea on what is going on or how to fix this?

This only happens with "Every tick based on real ticks" mode and Metatrader version is 5.00 build 2009.

Edit: forgot to mention that execution is set to "Without Delay".

 

Ill attempt to reproduce the issue  (for local cores), so here are some technical questions :

Is the test running on one asset ?

What is your date range ?

Are trades "deflected" by specific conditions ?(anything other than available margin)

Do trades rely on indicators that are external ?(not built in the ea code)

Is the position size static ?

 
Lorentzos Roussos:

Ill attempt to reproduce the issue  (for local cores), so here are some technical questions :

Is the test running on one asset ?

What is your date range ?

Are trades "deflected" by specific conditions ?(anything other than available margin)

Do trades rely on indicators that are external ?(not built in the ea code)

Is the position size static ?

Thank you for the interest.

The test is running on USDJPY on ForexTime.

2019.04.07 - 2019.04.13

No, the trades are not "deflected" by any other conditions.

Yes the indicator is external and brought in by the #resource directive. The indicator is basically a price level and does not change much. The EA decides based on the price relative to that level so it is important to use a simulation as close to real values as possible.

The position of the size is not static, it is calculated on the fly based on the available margin.

If it helps, the strategy tester seems to do every single test either right or with no trades at all.
 
Dimitrios Vezeris:

Thank you for the interest.

The test is running on USDJPY on ForexTime.

2019.04.07 - 2019.04.13

No, the trades are not "deflected" by any other conditions.

Yes the indicator is external and brought in by the #resource directive. The indicator is basically a price level and does not change much. The EA decides based on the price relative to that level so it is important to use a simulation as close to real values as possible.

The position of the size is not static, it is calculated on the fly based on the available margin.

If it helps, the strategy tester seems to do every single test either right or with no trades at all.

During the optimization or even on the same "optimized set" when it runs on its own (outside of optimization)?

 
Lorentzos Roussos:

During the optimization or even on the same "optimized set" when it runs on its own (outside of optimization)?

During the optimization. When it is run on its own it produces reasonable results.

 

Okay , ive run tests with MT5 builds 2007 , 2025  (non broker mt5's).
Its consistent across the board .
What is the reading for real ticks you received ? (i got "USDJPY : Real Ticks Start 2019.04.08 00:00:00")

I would try to start the test earlier and limit trading  beyond (only after) the real ticks date . (but i could not reproduce the issue , so its an idea) 

 
Lorentzos Roussos:

Okay , ive run tests with MT5 builds 2007 , 2025  (non broker mt5's).
Its consistent across the board .
What is the reading for real ticks you received ? (i got "USDJPY : Real Ticks Start 2019.04.08 00:00:00")

I would try to start the test earlier and limit trading  beyond (only after) the real ticks date . (but i could not reproduce the issue , so its an idea) 

I have a "USDJPY: ticks data begins from 2016.08.04 00:00", I'm not sure what is it or if it plays a role.

Thank you for your effort nonetheless.

 
Dimitrios Vezeris:

I have a "USDJPY: ticks data begins from 2016.08.04 00:00", I'm not sure what is it or if it plays a role.

Thank you for your effort nonetheless.

No its not it ,you have data that covers the "lead in" of the test anyway .

 

Back with a few more details and observations. It seems that, after some passes during the optimization, tests fail to start or give back any results.

The images show the Strategy Tester Settings, Optimization Results and Individual Test Results.

The Metatrader version is 5.00 build 2009 and only local agents were used for the optimization.

Settings

Graph



Results

Sample Test

 

Could be related to this error: https://www.mql5.com/en/forum/305738

Expert optimization is failing after some passes
Expert optimization is failing after some passes
  • 2019.03.05
  • www.mql5.com
Hello all, I'm facing a strange error...
Reason: