Can't backtest EA on Futures - page 4

 
Rashid Umarov #:

Think again carefully, taking into account my answer above.

Don't think you're smarter than developers who have been writing terminal and tester code for over 20 years.

I am suggesting you take rest, as you are saying wrong things. You have no idea who is smarter, and it has absolutely nothing to do with the topic. (And by the way I didn't insult anyone, please check the English expression "you are making a fool of yourself", it's by no way an insult).

If the required bars are not available for whatever reason, the starting date of testing is automatically shifted from the past to the present to provide the necessary amount of bars.

The fact is MT5 has currently a "past bars" limitations, which is NOT documented and it would be easy to improve that.

2025.08.22 14:48:45.271 Tester  "Shared Projects\MQL5\Shared\ServiceDesk\NoHistoryRequested.ex5" AVX2
2025.08.22 14:48:45.668 Tester  FUTNQU25: history data begins from 2025.03.12 00:00
2025.08.22 14:48:45.892 Tester  FUTNQU25: preliminary downloading of history ticks started, it may take quite a long time
2025.08.22 14:48:45.892 Tester  FUTNQU25: preliminary downloading of history ticks completed
2025.08.22 14:48:45.892 Tester  FUTNQU25: ticks data begins from 2025.05.09 00:00
2025.08.22 14:48:45.961 Core 02 agent process started on 127.0.0.1:3001
2025.08.22 14:48:45.961 Core 02 connecting to 127.0.0.1:3001
2025.08.22 14:48:46.467 Core 02 connected
2025.08.22 14:48:46.474 Core 02 authorized (agent build 5120)
2025.08.22 14:48:46.482 Tester  FUTNQU25,Daily (Broker-xyz): testing of Experts\Shared Projects\MQL5\Shared\ServiceDesk\NoHistoryRequested.ex5 from 2025.06.25 00:00 to 2025.06.30 00:00
2025.08.22 14:48:46.516 Core 02 common synchronization completed
2025.08.22 14:48:46.540 Core 02 FUTNQU25: ticks synchronized already [47 bytes]
2025.08.22 14:48:46.556 Core 02 MetaTester 5 started on 127.0.0.1:3001
2025.08.22 14:48:46.556 Core 02 cloud network mode is off
2025.08.22 14:48:46.556 Core 02 initialization finished
2025.08.22 14:48:46.556 Core 02 login (build 5120)
2025.08.22 14:48:46.556 Core 02 148988 bytes of account info loaded
2025.08.22 14:48:46.556 Core 02 1478 bytes of tester parameters loaded
2025.08.22 14:48:46.556 Core 02 196 bytes of input parameters loaded
2025.08.22 14:48:46.556 Core 02 62321 bytes of symbols list loaded (11245 symbols)
2025.08.22 14:48:46.556 Core 02 expert file added: Experts\Shared Projects\MQL5\Shared\ServiceDesk\NoHistoryRequested.ex5. 5732 bytes loaded
2025.08.22 14:48:46.556 Core 02 1012 Mb available, 12 blocks set for ticks generating
2025.08.22 14:48:46.556 Core 02 initial deposit 1000.00 USD, leverage 1:500
2025.08.22 14:48:46.556 Core 02 successfully initialized
2025.08.22 14:48:46.556 Core 02 73 Kb of total initialization data received
2025.08.22 14:48:46.556 Core 02 Intel Core i7-9750H  @ 2.60GHz, 16228 MB
2025.08.22 14:48:46.556 Core 02 FUTNQU25: symbol to be synchronized
2025.08.22 14:48:46.556 Core 02 FUTNQU25: symbol synchronized, 3720 bytes of symbol info received
2025.08.22 14:48:46.556 Core 02 FUTNQU25: load 31 bytes of history data to synchronize in 0:00:00.000
2025.08.22 14:48:46.556 Core 02 FUTNQU25: history synchronized from 2025.03.12 to 2025.06.30
2025.08.22 14:48:46.556 Core 02 FUTNQU25: ticks synchronization started
2025.08.22 14:48:46.556 Core 02 FUTNQU25: load 38 bytes of tick data to synchronize in 0:00:00.000
2025.08.22 14:48:46.556 Core 02 FUTNQU25: history ticks synchronized from 2025.06.25 to 2025.06.27
2025.08.22 14:48:46.556 Core 02 FUTNQU25: start time changed to 2025.07.01 00:00 to provide data at beginning
2025.08.22 14:48:46.556 Core 02 FUTNQU25,Daily: history cache allocated for 78 bars and contains 78 bars from 2025.03.12 00:00 to 2025.06.30 00:00
2025.08.22 14:48:46.556 Core 02 FUTNQU25,Daily: history begins from 2025.03.12 00:00
2025.08.22 14:48:46.556 Core 02 FUTNQU25,Daily (Just2Trade-MT5): generating based on real ticks
2025.08.22 14:48:46.556 Core 02 FUTNQU25,Daily: testing of Experts\Shared Projects\MQL5\Shared\ServiceDesk\NoHistoryRequested.ex5 from 2025.06.25 00:00 to 2025.06.30 00:00 started
2025.08.22 14:48:46.556 Core 02 final balance 1000.00 USD
2025.08.22 14:48:46.556 Core 02 OnTester result 0
2025.08.22 14:48:46.556 Core 02 FUTNQU25,Daily: 0 ticks, 0 bars generated. Environment synchronized in 0:00:00.043. Test passed in 0:00:00.039.
2025.08.22 14:48:46.556 Core 02 FUTNQU25,Daily: total time from login to stop testing 0:00:00.082 (including 0:00:00.043 for history data synchronization)
2025.08.22 14:48:46.556 Core 02 88 Mb memory used including 0.47 Mb of history data, 0 Mb of tick data
2025.08.22 14:48:46.556 Core 02 log file "D:\Apps\MT5\Alpari\Tester\Agent-127.0.0.1-3001\logs\20250822.log" written
2025.08.22 14:48:46.567 Core 02 connection closed

Firstly the EA didn't request any history data.

Secondly what is the the "requested number of bars" ? 1 year ? 100 bars ? In all cases switch from 2025.06.25 to 2025.07.01 changes NOTHING, it's still not 100 bars, it's still not 1 year and mainly the EA doesn't request them at all, so that seems rather arbitrary and undocumented for sure.


Please stop being offended, start to listen people, and try to improve MT5, I have no doubt MetaQuotes is able to do it, if they want.

Files:
 
Alain Verleyen #The fact is MT5 has currently a "past bars" limitations, which is NOT documented and it would be easy to improve that.

I agree and second this opinion and request.

Alain Verleyen #Firstly the EA didn't request any history data.

I also agree.

@Rashid Umarov, I understand MetaQuotes' reasoning with regard to Indicators that may need that amount of minimum bars for their functionality, but the EA may not be using indicators at all, and may even not be requiring so many bars of data.

It would have been a smarter decision to simply have the Strategy Tester issue a warning message and fall back to available data irrespectively. In fact there is already an example of such an approach, for example when the tester detects missing tick data (real ticks testing)—it issues a warning messages and falls back to an alternative method.

Let the EA's developer handle the discrepancy on their own instead of imposing a limit that may not even be relevant.

 

In short:

1. The theoretical user will demand "do as I want, you are limiting me". There are one and a half such users

2. The practicing developer understands that this cannot be done - the testing results may become non-reproducible due to the fact that the indicator values differ from different lengths of history. To find this out, it will be necessary to conduct research and spend time each time. This is called "shooting yourself in the foot".

As a result, you can get massive problems and accusations of incorrect tester operation out of the blue. This case is isolated and no one will shoot themselves in the foot for it.

A practical trader will solve the problem (by changing the testing timeframe/EA logic, or by creating a custom symbol) and move on. And the fighters for justice, who themselves are not interested in this case at all, will remain on the barricades. Simply because the developers must do as they want. It is so easy - to add a couple more settings, so that the system becomes even more difficult to use.

I think it is much easier to create a profitable trading system for trading futures on daily bars, having a history of several weeks. It is logical.

 
Rashid Umarov #:

In short:

1. The theoretical user will demand "do as I want, you are limiting me". There are one and a half such users

2. The practicing developer understands that this cannot be done - the testing results may become non-reproducible due to the fact that the indicator values differ from different lengths of history. To find this out, it will be necessary to conduct research and spend time each time. This is called "shooting yourself in the foot".

As a result, you can get massive problems and accusations of incorrect tester operation out of the blue. This case is isolated and no one will shoot themselves in the foot for it.

A practical trader will solve the problem (by changing the testing timeframe/EA logic, or by creating a custom symbol) and move on. And the fighters for justice, who themselves are not interested in this case at all, will remain on the barricades. Simply because the developers must do as they want. It is so easy - to add a couple more settings, so that the system becomes even more difficult to use.

I think it is much easier to create a profitable trading system for trading futures on daily bars, having a history of several weeks. It is logical.

I will bookmark this one for the record. Masterpiece of arrogance and sophism, it says a lot.

"Abandon all hope ye who enter here"

 
Rashid Umarov #:

In short:

1. The theoretical user will demand "do as I want, you are limiting me". There are one and a half such users

2. The practicing developer understands that this cannot be done - the testing results may become non-reproducible due to the fact that the indicator values differ from different lengths of history. To find this out, it will be necessary to conduct research and spend time each time. This is called "shooting yourself in the foot".

As a result, you can get massive problems and accusations of incorrect tester operation out of the blue. This case is isolated and no one will shoot themselves in the foot for it.

A practical trader will solve the problem (by changing the testing timeframe/EA logic, or by creating a custom symbol) and move on. And the fighters for justice, who themselves are not interested in this case at all, will remain on the barricades. Simply because the developers must do as they want. It is so easy - to add a couple more settings, so that the system becomes even more difficult to use.

I think it is much easier to create a profitable trading system for trading futures on daily bars, having a history of several weeks. It is logical.


So, you are saying that, because users are smart enough to find work-around solutions, it is totally justified for ill-designed, sub-par, crippled software. Brilliant! 🙃

 
I have followed this entire discussion and, honestly, I think Rashid made a very valid point. The tester is designed with the broader user base in mind, and the requirement for prior history exists to protect the consistency of results. For most users this is actually an advantage, because it prevents misleading backtests and avoids situations where the tester would be blamed for producing unreliable results. I find it correct and logical that MetaQuotes prioritizes the reliability of the tool over isolated cases.

That said, I also understand what Fernando and Alain mentioned. It would indeed be useful if the terminal displayed a more explicit message when this rule is applied, so that users do not mistake it for a bug. Communication in that sense could certainly be improved.

What does surprise me is to see two such experienced users like Fernando and Alain raising such strong concerns over something that, after so many years programming in MQL and working with the tester, they themselves had not noticed until now. That, in itself, shows that this is not a critical issue. If it went unnoticed for so long even by highly experienced users, then clearly it does not block day-to-day development.

In summary, this is not a bug. It is documented and has a clear technical rationale. For the general audience the limitation is positive because it ensures consistency. For advanced developers there are simple workarounds such as testing on H1 or creating custom symbols. The only improvement that might be worth considering is clearer messaging in the terminal when this happens.

Therefore I believe MetaQuotes is right to maintain this design as it is, and to dedicate their efforts to areas of greater impact on the platform.

In the end, everything has a reason to be, whether we agree with it or not, and often even when we do not fully understand it. In a perfect world the tester would give us both consistency and total flexibility, but in practice design choices always involve trade-offs. And since we are all developers, we should be the first to recognize that. We deal with it every day: when we design a strategy and want it to run reliably across different brokers, with different symbols, schedules, and market conditions, we inevitably make adjustments and concessions so that our work remains usable in a wide range of circumstances and for many different users. The same principle applies here. That is why I find it difficult to understand the lack of empathy towards MetaQuotes, when some expect absolute perfection even in a case where, as now, the explanation is both clear and logical: ensuring consistency of results.

And yes, I can already imagine the counterargument: "precisely because we are developers, we cannot understand why something so simple is not fixed". But that is exactly the point. As developers, we should know better than anyone that what looks simple in isolation can have far-reaching implications once it becomes part of a complex system used by millions of people. That two users may not obtain the same result in a test is no small matter, as it would cause greater harm than what is supposedly being "fixed".

Of course this is just my personal opinion, without any attachment or interest. I am not here to take sides, only to share my view as someone with enough perspective.
 
Alain Verleyen #Secondly what is the the "requested number of bars" ? 1 year ? 100 bars ? In all cases switch from 2025.06.25 to 2025.07.01 changes NOTHING, it's still not 100 bars, it's still not 1 year and mainly the EA doesn't request them at all, so that seems rather arbitrary and undocumented for sure.

Alain, your log clearly illustrates what feels like a poor user experience: the start date is shifted to 2025.07.01, but that still results in zero ticks and zero bars, and the terminal does not explain why. I fully understand why this looks arbitrary, and I agree with you that the documentation does not make this obvious.

To be precise, the documentation does mention this behavior, but not in a very clear way. In the section Testing Features > Multi-Currency Testing it states:

"For the timeframes D1 and below, the minimum volume of the downloaded history is one year. If the required bars are not available for whatever reason, the starting date of testing is automatically shifted from the past to the present to provide the necessary amount of bars".

So technically it is documented, but the problem is that it is hidden under the multi-currency section, when in fact this is a general rule that applies to all tests, and it does not explain the exact logic of how the number of bars or length of history is decided in each case. This is why it gives the impression of being arbitrary.

If I had to suggest an improvement, I would make this rule explicit in a dedicated section of the Strategy Tester documentation, clearly stating that the tester always reserves a buffer of past bars before the chosen start date. I would also have the terminal itself display a clearer message when shifting the date, for example: "Start date adjusted due to insufficient history.". That way users immediately understand what is happening and why.

So I would say you are absolutely right about the messaging and the way the documentation is presented, it could and should be clearer so that users do not mistake this for a bug. At the same time, the underlying logic is intentional and is meant to ensure consistency of results across users. Without such a safeguard, two developers with different amounts of history loaded locally could end up with completely different backtest results, which would be far worse.

Forum on trading, automated trading systems and testing trading strategies

Can't backtest EA on Futures

Anton, 2025.08.22 12:51

The test period is from 2025.06.25 to 2025.06.30, and the test start time is shifted to 2025.08.19.

I think the documentation here doesn't describe the bar history requirement well enough. Minimum 100 bars before the test start date is a very old requirement. The article is from 2011, and this tester behavior is indeed very old.


 
Finally, I would like to close with a general observation. This thread has included many valuable technical points, but at times the tone has moved away from the purely technical and into personal ground. That never helps. It raises the temperature of the debate, creates unnecessary tension, and distracts from the real issue we are trying to solve.

We all share responsibility for keeping discussions constructive. Everyone here has years of experience and valid perspectives, and that experience is most useful when expressed with respect. If we keep the conversation professional and focused on the technical arguments, even when we strongly disagree, the result is a dialogue that benefits the community, the platform, and ultimately ourselves as developers.

In the end, I believe the community expects more from us. We are here to exchange knowledge and improve the tools we use, not to put on a spectacle. Keeping discussions respectful and professional is the best way to achieve that.

Thank you, and apologies if I have been a bit long-winded or repetitive. That is simply my way of expressing myself, I do not like to leave loose ends and I always try not to.
 
Miguel Angel Vico Alba #:

Alain, your log clearly illustrates what feels like a poor user experience: the start date is shifted to 2025.07.01, but that still results in zero ticks and zero bars, and the terminal does not explain why. I fully understand why this looks arbitrary, and I agree with you that the documentation does not make this obvious.

To be precise, the documentation does mention this behavior, but not in a very clear way. In the section Testing Features > Multi-Currency Testing it states:

"For the timeframes D1 and below, the minimum volume of the downloaded history is one year. If the required bars are not available for whatever reason, the starting date of testing is automatically shifted from the past to the present to provide the necessary amount of bars".

So technically it is documented, but the problem is that it is hidden under the multi-currency section, when in fact this is a general rule that applies to all tests, and it does not explain the exact logic of how the number of bars or length of history is decided in each case. This is why it gives the impression of being arbitrary.

If I had to suggest an improvement, I would make this rule explicit in a dedicated section of the Strategy Tester documentation, clearly stating that the tester always reserves a buffer of past bars before the chosen start date. I would also have the terminal itself display a clearer message when shifting the date, for example: "Start date adjusted due to insufficient history.". That way users immediately understand what is happening and why.

So I would say you are absolutely right about the messaging and the way the documentation is presented, it could and should be clearer so that users do not mistake this for a bug. At the same time, the underlying logic is intentional and is meant to ensure consistency of results across users. Without such a safeguard, two developers with different amounts of history loaded locally could end up with completely different backtest results, which would be far worse.


Miguel, I am not sure why you feel the needs to intervene after all is done finally, anyway let me answer.

I clearly proved this Tester behaviour is NOT documented. Though it's not even what matters. A simple user reported an issue, 2 moderators checked and confirmed. Then, at my request by the way, someone of MetaQuotes answered... with an accusatory tone against the OP. That's clearly not acceptable, is it ? It's clear this employee doesn't even take the time to read the topic as he requested info which are already there.

Then an other employee, higher in MetaQuotes hierarchy comes, and he basically attacked the 2 moderators, and said that the OP just has to be smarter and find workaround. Again, that's clearly not acceptable for me, is it for you ? 

Maybe I should have done things differently, but I don't think my messages here are the problem. The OP provided all needed information, Fernando confirmed, and from there I clearly demonstrated the issue : an unexpected behaviour, with code and logs ; asking for explanations, I got personal attack in response. In my opinion, a software is done to help people do what they want, and not the reverse (where people have to adapt what they want to do to met the software design and limitations). 

In your message you take what was said by MetaQuotes as a truth, it's nota truth : "the underlying logic is intentional and is meant to ensure consistency of results across users. Without such a safeguard, two developers with different amounts of history loaded locally could end up with completely different backtest results". Nobody suggested to make changes leading to inconsistencies, this is what I called a sophism. 

This topic has clearly demonstrated how MetaQuotes is contemptuous with people, they just don't care about users, about moderators doing free work for them, and the worst thing maybe, they don't care about the quality of their software. They know better than everyone. I guess it's where success and monopole lead.

For me the topic is closed.

 
Alain Verleyen #:

Miguel, I am not sure why you feel the needs to intervene after all is done finally, anyway let me answer.

I clearly proved this Tester behaviour is NOT documented. Though it's not even what matters. A simple user reported an issue, 2 moderators checked and confirmed. Then, at my request by the way, someone of MetaQuotes answered... with an accusatory tone against the OP. That's clearly not acceptable, is it ? It's clear this employee doesn't even take the time to read the topic as he requested info which are already there.

Then an other employee, higher in MetaQuotes hierarchy comes, and he basically attacked the 2 moderators, and said that the OP just has to be smarter and find workaround. Again, that's clearly not acceptable for me, is it for you ? 

Maybe I should have done things differently, but I don't think my messages here are the problem. The OP provided all needed information, Fernando confirmed, and from there I clearly demonstrated the issue : an unexpected behaviour, with code and logs ; asking for explanations, I got personal attack in response. In my opinion, a software is done to help people do what they want, and not the reverse (where people have to adapt what they want to do to met the software design and limitations). 

In your message you take what was said by MetaQuotes as a truth, it's nota truth : "the underlying logic is intentional and is meant to ensure consistency of results across users. Without such a safeguard, two developers with different amounts of history loaded locally could end up with completely different backtest results". Nobody suggested to make changes leading to inconsistencies, this is what I called a sophism. 

This topic has clearly demonstrated how MetaQuotes is contemptuous with people, they just don't care about users, about moderators doing free work for them, and the worst thing maybe, they don't care about the quality of their software. They know better than everyone. I guess it's where success and monopole lead.

For me the topic is closed.

Alain, thank you for your reply. I know you consider the topic closed, but I think it is only fair to clarify a few points before leaving it behind, just to put the dots on the i's.

I fully understand your position and I also agree that some of the replies from MetaQuotes could and should have been expressed in a more constructive way. On that point there is no doubt.

That said, I do not think it is entirely accurate to say that nobody suggested changes that could lead to inconsistencies. Fernando explicitly wrote: "It would have been a smarter decision to simply have the Strategy Tester issue a warning message and fall back to available data irrespectively".

I understand the motivation behind this idea, but technically such a change would allow two users with different amounts of historical data locally to obtain different backtest results. That is precisely what the current design is intended to prevent.

It was in response to this suggestion that Rashid replied with his now well-known comment: "Think again carefully… Don't think you're smarter than developers who have been writing terminal and tester code for over 20 years". The phrasing was clearly unfortunate, but the underlying point was that what may appear simple and logical in isolation could, in practice, undermine consistency and reproducibility across millions of users.

It is also worth noting that Anton himself acknowledged that the documentation on this point is not clear or well written ("I think the documentation here doesn't describe the bar history requirement well enough. Minimum 100 bars before the test start date is a very old requirement. The article is from 2011, and this tester behavior is indeed very old."). So I do not think anyone is trying to sweep the issue under the rug. What seems to have happened is that the discussion shifted: what could have remained a straightforward request for clearer documentation gradually turned into a debate about whether the tester’s underlying logic should be redesigned. That is when things became more confused than they needed to be.

So while I agree with you that the tone used by MQ was not ideal, the reasoning behind their answer makes sense in this specific context. From their perspective, allowing flexibility at the cost of reproducibility is not an acceptable trade-off.

In any case, I think the key point where we all agree is that the messaging and documentation around this behavior should be clearer. If the terminal explicitly explained that the start date was shifted due to the need for a one-year history buffer, users would not mistake it for a bug in the first place. That, I believe, is a constructive improvement worth pushing for.