The backtest won. The account lost.

We build trading robots. While building three of them for gold, Relay, Assay and Goldworks, we ran thousands of backtests and studied other products by watching how they trade. We learned that most robots which lose money live were never going to win. Some were built to look good in a screenshot. Many fooled their own developer first. Often nobody had to lie at all.

This post collects all of it, from the oldest trick to the newest machine-learning mistake, with the figures we measured ourselves. No product is named. What matters is the trick, so you can recognise it anywhere.

1. Picking up coins in front of a steamroller

Martingale doubles the trade size after every loss, so the first win pays back everything plus a little. Grid opens another position each time price moves against the last one, and closes the whole basket on a small bounce. Both win almost every week, so the curve rises smoothly for months.

The catch is arithmetic. Each doubling makes the next loss twice as large, and the account can only pay a limited number of them. Sooner or later an ordinary run of losses, the kind every market produces, asks for more than the account has. Months of coins are returned in one afternoon, with the rest of the deposit. It is not a question of whether, only when.

2. 95% wins, and still losing money

The easiest way to sell a robot is a high win rate. The easiest way to make one is a far stop loss and a near take profit. Price touches a near target most of the time, so the robot wins 95 trades out of 100. But each loss is 25 times the size of a win. Ninety-five wins of 10 make 950, five losses of 250 take 1,250, and the account is down 300 while the advertisement says "95% accurate".

What we measured. A dip-buying family we studied reached a win rate of 82 to 88 percent mechanically, and over ten years made between minus 240 and plus 269 dollars, statistically nothing. On one of our own robots, an exit that locked in small gains raised the win rate in all 68 settings we tried and lost money in all 68. An indicator's own panel showed 53.8% wins; counted honestly, with break-even exits not booked as wins, it was 21.1%.

3. Date fixing: the window is the product

Every strategy has good years and bad years. A backtest can start and stop on any date, so a seller can simply show the stretch where the robot happened to work, and leave out the years before and after. The developer's own bad period disappears the same way: start the test the month after it.

What we measured. On one of our robots, the same settings started on 1 January instead of 27 January 2026 made 13,292 instead of 1,954 dollars. After an improvement, 2026 from 1 January showed plus 17,329; the same year from 1 February showed minus 447. Nothing about the robot changed, only the first day of the test.

4. A small edge, blown up to the sky

A robot with a thin real edge can be made to look spectacular. Raise the risk per trade and let the profits compound, and a modest result becomes "plus 1,800 percent". The same size that builds the curve destroys it at the first ordinary losing streak; the screenshot is taken before.

What we measured. One of our own robots showed plus 1,849 percent in a year on a 1,500-dollar account, with a worst equity drawdown of 61.6 percent behind it. The percentage stayed between 1,506 and 1,849 at every deposit from 1,500 to 25,000 dollars: it measured the lot-size rule, not a bigger edge.

The best of a hundred coins. Test enough variations and one of them will look brilliant by pure chance. An optimizer does exactly this, thousands of times a minute. We tested all 420 combinations of two settings on each of two years: seven of the ten best settings for 2025 fell apart in 2026, earning 790 to 1,265 where our own values earned 2,833. When we searched thousands of trading rules, the best real rule was no better than the best rule found in shuffled, meaningless data. And a setting that looks perfect can sit next to a cliff: on one of our robots a size step of 400 earned 12,376, 350 earned 4,879, 300 earned 12,522 and 250 earned 4,938.

A robot that knew the past. One product we studied, by its behaviour only, won 81.2 percent of its trades where a fair game would give 38 percent. Moving the same trades by one day dropped it to 39.7 percent, and its timing broke down completely after a date in July 2026. It behaved less like a strategy than like a robot given the answers for the history it was tested on.

5. Machine learning: brilliant on known data, lost on new data

This one needs no dishonesty at all. A flexible model shown the past will learn the past, including its noise, perfectly. On data it has already seen it is almost never wrong. On the next month, which it has not seen, it is often no better than a coin. "AI" on the box is not evidence of anything.

What we measured. A boosted model fitted to our own trades scored a perfect 1.000 on the data it learned from and between minus 0.02 and minus 4.11 on data it had not seen. Its ability to tell good trades from bad on new data scored 0.38 to 0.51, where 0.50 is a coin flip. On Goldworks, an entry filter that multiplied profit 2.7 times on its training data lost money on both test periods. A price model that looked strong on known bars was flat on new ones, and its results changed sign with the holding time.

6. The tester can be told what to say

The MetaTrader Strategy Tester is honest, but it answers the question it is asked. Cheaper modelling, missing costs and perfect execution all make the same robot look better.

A product we studied advertised a profit factor of 6.24. Re-run on the same machine it was below 1, and over 13 live weeks it made 1.94: in the backtest every loss was exactly one unit of risk, live the average loss was more than three. The backtest belonged to one download of tick data, not to the strategy.

7. The mistakes we made ourselves

The honest kind is the most common, so here are ours. We publish them because each one would have reached a buyer if we had not caught it.

• We quoted the wrong drawdown. This very week, on our own sales images, we first read the "Maximal" drawdown row (19.9%) instead of the "Relative" one (29.6%), the true worst percentage. It was corrected before release.
• Our research tool read the future. A shortcut calculator said an idea improved every strategy. In the real tester it made 576 dollars where the shipped rules made 15,801. Three studies built on it were withdrawn.
• Comments that lied about the code. In one day six defects were introduced and caught, three of them in the fix for the previous three, and two were comments describing what the code did not do. None was found by a backtest; all were found by reading.
• A result that was one October. More than once, a promising rule turned out to be one good month. It was rejected because the next year disagreed.

8. How it empties accounts

9. Before you pay: a ten-point check

10. What we did differently with Relay

We wrote this post because Relay is sold next to products that use these tricks, and the only answer we have is to show our work. Here is each trick, and what Relay does instead.

Trading leveraged products carries a substantial risk of loss and is not suitable for every investor. Past performance, including any backtest, does not guarantee future results. This post is education, not financial advice.