I kept seeing EAs advertised with backtests showing thousands of percent per year, and honestly I stopped believing any of them. So instead of arguing about it, I decided to just test them myself.
Over about three weeks I downloaded more than 800 of the most popular EAs from the Market and ran every single one on real ticks. Not demo data — real ticks, with the spread that actually shows up. I reset the balance to the same amount every year so compounding wouldn't inflate the numbers. My Mac was not happy about this. One EA alone spat out something like 60GB of temporary files and filled my whole disk before I caught it.
Here's what surprised me.
Only about one in five EAs even finished cleanly with a profit. The rest either made no trades, ran forever until they timed out, or just crashed on load. And the higher I went up the popularity ranking, the more everything turned into gold scalpers — most of which melted the account the instant I switched from demo bars to real ticks.
Then I found the thing I wasn't expecting. I started checking the build date of each product that survived. A lot of the good-looking ones were compiled after the period I was testing. Think about what that means: if you already know how the market moved, of course you can make a curve that fits it perfectly. That's not an edge, it's hindsight. And the flashier the sales page, the more likely it was one of these.
So I threw out everything built after my test window, kept only the ones that were genuinely tested on an unknown period, and then asked one more question: does it have a real, live track record? Not a backtest — an actual account.
Zero of them did.
I'm not saying every market EA is garbage, and this wasn't a scientific paper — I could be wrong on some of the edge cases. But the pattern was hard to ignore: the products with the loudest demo numbers had the least live evidence behind them.
The lesson I took away is that a backtest is only good for rejecting an EA. If it melts on real ticks, or it was built after the test window, you can drop it. But a great backtest, on its own, proves nothing about live trading. The only things worth trusting are measured real-tick results and a live account that actually held up.
That's why I gave up on posting flashy demo numbers for my own EAs and just publish the measured real-tick figures instead — good and bad.
