Highly inconsistent results in backtesting

To add comments, please log in or register
COSMOS4U - D. Vezeris & Co, L.P.
2614
Dimitrios Vezeris  

We run backtests with "Slow complete algorithm" for a range of inputs in "Every tick based on real ticks".
In the optimization results we click a single test to run and we get totally different results.

Moreover when we use other local network agents the "Slow complete algorithm" results are different from the "Slow complete algorithm" results obtained previously through the local cores.

Does anyone have any idea on what is going on or how to fix this?

This only happens with "Every tick based on real ticks" mode and Metatrader version is 5.00 build 2009.

Edit: forgot to mention that execution is set to "Without Delay".

Lorentzos Roussos
29923
Lorentzos Roussos  

Ill attempt to reproduce the issue  (for local cores), so here are some technical questions :

Is the test running on one asset ?

What is your date range ?

Are trades "deflected" by specific conditions ?(anything other than available margin)

Do trades rely on indicators that are external ?(not built in the ea code)

Is the position size static ?

COSMOS4U - D. Vezeris & Co, L.P.
2614
Dimitrios Vezeris  
Lorentzos Roussos:

Ill attempt to reproduce the issue  (for local cores), so here are some technical questions :

Is the test running on one asset ?

What is your date range ?

Are trades "deflected" by specific conditions ?(anything other than available margin)

Do trades rely on indicators that are external ?(not built in the ea code)

Is the position size static ?

Thank you for the interest.

The test is running on USDJPY on ForexTime.

2019.04.07 - 2019.04.13

No, the trades are not "deflected" by any other conditions.

Yes the indicator is external and brought in by the #resource directive. The indicator is basically a price level and does not change much. The EA decides based on the price relative to that level so it is important to use a simulation as close to real values as possible.

The position of the size is not static, it is calculated on the fly based on the available margin.

If it helps, the strategy tester seems to do every single test either right or with no trades at all.
Lorentzos Roussos
29923
Lorentzos Roussos  
Dimitrios Vezeris:

Thank you for the interest.

The test is running on USDJPY on ForexTime.

2019.04.07 - 2019.04.13

No, the trades are not "deflected" by any other conditions.

Yes the indicator is external and brought in by the #resource directive. The indicator is basically a price level and does not change much. The EA decides based on the price relative to that level so it is important to use a simulation as close to real values as possible.

The position of the size is not static, it is calculated on the fly based on the available margin.

If it helps, the strategy tester seems to do every single test either right or with no trades at all.

During the optimization or even on the same "optimized set" when it runs on its own (outside of optimization)?

COSMOS4U - D. Vezeris & Co, L.P.
2614
Dimitrios Vezeris  
Lorentzos Roussos:

During the optimization or even on the same "optimized set" when it runs on its own (outside of optimization)?

During the optimization. When it is run on its own it produces reasonable results.

Lorentzos Roussos
29923
Lorentzos Roussos  

Okay , ive run tests with MT5 builds 2007 , 2025  (non broker mt5's).
Its consistent across the board .
What is the reading for real ticks you received ? (i got "USDJPY : Real Ticks Start 2019.04.08 00:00:00")

I would try to start the test earlier and limit trading  beyond (only after) the real ticks date . (but i could not reproduce the issue , so its an idea) 

COSMOS4U - D. Vezeris & Co, L.P.
2614
Dimitrios Vezeris  
Lorentzos Roussos:

Okay , ive run tests with MT5 builds 2007 , 2025  (non broker mt5's).
Its consistent across the board .
What is the reading for real ticks you received ? (i got "USDJPY : Real Ticks Start 2019.04.08 00:00:00")

I would try to start the test earlier and limit trading  beyond (only after) the real ticks date . (but i could not reproduce the issue , so its an idea) 

I have a "USDJPY: ticks data begins from 2016.08.04 00:00", I'm not sure what is it or if it plays a role.

Thank you for your effort nonetheless.

Lorentzos Roussos
29923
Lorentzos Roussos  
Dimitrios Vezeris:

I have a "USDJPY: ticks data begins from 2016.08.04 00:00", I'm not sure what is it or if it plays a role.

Thank you for your effort nonetheless.

No its not it ,you have data that covers the "lead in" of the test anyway .

COSMOS4U - D. Vezeris & Co, L.P.
2614
Dimitrios Vezeris  

Back with a few more details and observations. It seems that, after some passes during the optimization, tests fail to start or give back any results.

The images show the Strategy Tester Settings, Optimization Results and Individual Test Results.

The Metatrader version is 5.00 build 2009 and only local agents were used for the optimization.

Settings

Graph



Results

Sample Test

lippmaje
689
lippmaje  

Could be related to this error: https://www.mql5.com/en/forum/305738

Expert optimization is failing after some passes
Expert optimization is failing after some passes
  • 2019.03.05
  • www.mql5.com
Hello all, I'm facing a strange error...
To add comments, please log in or register