OnTester Help re: MT5 Optimisations Not Accurate

New comment
 

Hi All,

I know this thread has been bashed a number of times, but what's the general consensus on getting reproducible backtest results from the MT5 optimisation outputs?

1 - I've been running our multi-timeframe expert which we developed in house perfectly with regular backtests; the code is good and all functions, even in long period backtests (20 years), are relatively consistent in live performance. (Not using optimised results)

2 - Then when testing input sets derived from optimisations, it doesn't matter what size, or data set, any set of inputs the optimisation produces simply DO NOT perform accurately at all when testing them by running backtests after the optimisation discovered them.

basic example of how ridiculous the conflict is -

an optimisation discovered input set might state that the total number of trades conducted in a set over say, 10 years, is 1000, the maximum drawdown is 15%, profit factor is 3.5, then, manual backtests will show the same set of results completely tanking within 15-30 or so VERY unreliable trades.

  1. I've been at this now for several months of optimisation, after our algo has had more than 12 months design and development
  2. I've recently even gone as far as to code in some additional custom criteria in the OnTester function for improved profitability methods
  3. I've also been running and trying optimisations on all sorts of systems - Using our local machine, our network farm of virtualised servers and also even using the paid agents network when we've wanted an expedited optimisation.

Basically, for our purpose, something stinks with MT5s optimisation...

  • Does anyone have any suggestions around well known caveats of the fast-genetic optimisation?
  • Are there any well-known issues of reliability?
  • Are there any well-known fixes?

I've also read somewhere that MT4 optimisation is more reliable in some instances, in terms of reproducible performance.

Having gone through all avenues, and amending our approach to optimisation several times by way of a Root Cause Analysis to evaluate what potentially is causing the issues (be it Code, Process, Exchange Data, System Hardware, etc), we're stumped.

If anyone has any good suggestions for 3rd party optimisation software for MT5/MT4, by all means, this would be well received.

Our next steps (if this all fails) is scrapping MQL5 altogether and moving the system to Python, and running optimisation using Quandl or something similar; but, I'm not ready to divert our existing development roadmap that much at this stage.

  • Anyone with any suggestions is welcome to weigh in.
  • Any offers from community developers for OnTester function fixes are also welcome (Devs receiving source need to sign an NDA)

Cheers,

Iain

Background: Process Engineering / Systems Analyst / Enterprise Architect / Applied Finance

     

    I didn't read all your message, too long, with no data.

    Though I had similar issue, the problem was that when using "Run single backtest" command from the optimization results, some parameters were not correctly transmitted to the "Inputs". You could start by checking that if not yet done.

     

    MT5 has much more "bugs" when optimizing than MT4, that's true, but that's also the cost of much faster performance.

    There are multiple things you can consider to verify how reproducible is a result, when using a new set of indicator I ALWAYS test the values they retrieve in a visual test and a non-visual test (sometimes they can be inconsistent, but not always because they are repainters, it usually happens because of incorrect buffer updates)

    Also, check that your previous optimizations' cache doesn't interfer with new results. Usually changing one input (even a string that does nothing) can avoid that as a quick fix.

    And if you use multi timeframe, check how do the candles of higher/lower TFs look (because they may not adjust to reality). I personally do multi-symbol backtests, and the results can vary depending of the symbol that you choose because they are coordinated differently


    I've been doing this for a few months and my results are now pretty reliable, it really takes time to "master" all the little things, and still I sometimes find more flaws (like the effect of gaps in data). Depending on how much you plan to optimise things, it could be worth for you to use MT4 as a quick-one-time-thing. If you need to do that regularly, take some time to get reliable results in MT5 because it can be over 10 times faster than MT4 (at least with my current setup)

    New comment
    Reason: