bld509: why would 2 optimization results differ???

New comment
 

Hello

I run a optmization test on Friday (2014-02-14) and recorded results.

I then ran the same optimization test today and recorded results and they were different???

Why??

  • The settings are exactly the same, and by the way - I save/load them from a .set file.
  • The from and to dates are the same and they do not include today's date
  • Spread, Period, EA params... everything is the same, I did not touch anything since Friday!

... and yet the results differ... not by much... but still!

This is supposed to be a BLACK BOX - you feed it THE SAME DATA and it should give you back the SAME RESULT!!!

Here's the data from Friday (copied into a spread sheet):



... and this is what the results looks like today:


 
4x4ever:

Hello

I run a optmization test on Friday (2014-02-14) and recorded results.

I then ran the same optimization test today and recorded results and they were different???

Why??

  • The settings are exactly the same, and by the way - I save/load them from a .set file.
  • The from and to dates are the same and they do not include today's date
  • Spread, Period, EA params... everything is the same, I did not touch anything since Friday!

... and yet the results differ... not by much... but still!

This is supposed to be a BLACK BOX - you feed it THE SAME DATA and it should give you back the SAME RESULT!!!

It is if you understand what the inputs are . . . please read this thread, from start to finish: https://www.mql5.com/en/forum/148577
 

Thank you RaptorUK,

...what I understand from that thread is that generally it is "a very good idea" to disconnect from the broker when running a back-test in the tester?

In particular you ABSOLUTELY MUST disconnect from the server when:

  • your test pair is not XXX/AccountCurrency; or
  • Spread is set to "Current"; or
  • your model is "Every Tick" and your EA is opening trades on a per-tick basis (ie. mid-candle)

However this is not the case with my EA and my test:

1. I am testing with EURUSD and AccountCurrency is USD

2. Spread is 5

3. Model is Control points, Period is Daily and trades are only opened at mid-night, when new daily candle was just formed

Also, my "FROM - TO" period does not include today's date ... so that's one more thing where connection to the broker could screw things up... but not in my case.


Based on the above - is there any reason why the tests between Friday and today would be different?

And did I miss anything in my summary of that thread's info?

 

would pasting the contents of the tester/lasttest.chr file in here help?

I can save it, then run the same test tomorrow and if the results differ again - compare the two chr files to see if there's any differences in there...

 
4x4ever:

Thank you RaptorUK,

...what I understand from that thread is that generally it is "a very good idea" to disconnect from the broker when running a back-test in the tester?

In particular you ABSOLUTELY MUST disconnect from the server when:

  • your test pair is not XXX/AccountCurrency; or
  • Spread is set to "Current"; or
  • your model is "Every Tick" and your EA is opening trades on a per-tick basis (ie. mid-candle)

However this is not the case with my EA and my test:

1. I am testing with EURUSD and AccountCurrency is USD

2. Spread is 5

3. Model is Control points, Period is Daily and trades are only opened at mid-night, when new daily candle was just formed

Also, my "FROM - TO" period does not include today's date ... so that's one more thing where connection to the broker could screw things up... but not in my case.


Based on the above - is there any reason why the tests between Friday and today would be different?

And did I miss anything in my summary of that thread's info?

I think your synopsis is fair, thank you for reading the thread

If you disconnect from the Broker for all runs there can be no possibility for data to change, including history and symbol data . . . even if your test doesnt include today . . . incidentally it cannot, your Broker can still "update" other history for past days . . .

The thing about using an XXXYYY symbol where YYY is your Account currency is that in this situation the tickvalue is a constant, otherwise it changes tick by tick, if you then use tickvalue in your position size calculations you have another variable.
 

hmmm... it must be the swap.... becasue I compared dumps from both test runs... and I can see that open and close prices are exactly the same in Friday's test and today's test and yet the profit for each trade is just a tiny bit different. So the only thing that's left is the Swap (since Lots are also the same of course...).

It probably uses live SWAP rates in the tester.... just like many other things in the tester which come from the current live data...

 
4x4ever:

would pasting the contents of the tester/lasttest.chr file in here help?

I can save it, then run the same test tomorrow and if the results differ again - compare the two chr files to see if there's any differences in there...

You should print as many possible parameters that can affect your EAs calculations as possible in Init() and in deinit() then when you have 2 runs that are different you can compare the date from init() and deinit() . . . print as much as you can think of, spread, tickvalue, swaps, etc
New comment
Reason: