Twelve mistakes an AI-written expert advisor makes that you cannot see in the code
The scene has become routine, in the Freelance section here and in my own inbox: a rule in your head, a prompt to a language model, and three minutes later an expert advisor that compiles and wins in the strategy tester. The mistakes in that EA are rarely in the syntax. They are in what the code assumes about the market: when a piece of information exists, what a fill costs, how a broker counts, when a session really begins. I have built every one of these mistakes myself over the years. The difference now is the speed at which a model builds them, and that nobody is watching while it does.
This is the short version of a reference text on my site. It sorts the mistakes into three groups and ends with the twelve-question checklist I run before every client project.
Group 1: the backtest lies
Information from the future. Never crudely, never tomorrow's close. But a quantity defined for the day, read at a point where the day has not got that far yet. In generated MQL5 it looks like iHigh(_Symbol, PERIOD_D1, 0) used as "the day's high" at 10 a.m., or an opening range used as a filter from the first minute, or centred indicators (TMA bands, ZigZag) that redraw past candles. In my own episode base four context features had this problem despite a review routine: 60,129 of 123,193 episodes were affected, and a model's forecast spread fell from +0.726 R to +0.191 R after the fix. A client case from September 2026: an AI-written skeleton called TMA bands with HalfLength 32 on H4. Three arrows in seven weeks, all neatly at the band crossings, all hindsight.
The trigger comes late, the entry stays early. A filter waits for a confirmation, volume for instance. The condition becomes true later, but the entry price stays at the original level. A setup in the US midday lull showed +0.90 R at a 60% hit rate with a volume gate and −0.11 R at 29% without it. The gate had shifted the trigger by a median of 100 minutes while the entry stayed booked at the old level.
Fills that never happened. A bar-based simulator has to guess what happened inside a bar, and a generated one guesses in the strategy's favour, not by intent but because the optimistic variant is the simpler one. A doji pattern on the 2-minute chart showed +0.07 R net at t = 16 on real data. A synthetic random walk sent through the same bracket chain produced +0.16 R gross for the same pattern. The edge was a property of the chain.
Costs that are missing or flat. The number that decides before the backtest is the spread relative to the stop distance. A 0.80-point spread looked small on a Swiss index until it turned out to be about a quarter of the risk per trade.
A hit rate without a yardstick. With a stop twice as far away as the target, 67% is exactly what a coin flip delivers: stop divided by (stop plus target).
Two years are one regime, not proof. Six breakout setups over eleven years: 2015 to 2017 produced +15 R at a drawdown of −449 R, the eight years after that +621 R per year at −136 R. Same rules, two distributions.
Group 2: the code is right, the market is different
No backtest finds these, because the backtest shares them.
- Session hours and daylight saving. if(Hour() == 15) is correct until Europe and the US change their clocks on different weekends. Two or three times a year the US open shifts by an hour relative to server time, for two to three weeks.
- Points, pips, tick value. At one broker an index point is ten broker points, at the next a hundred. Every point-based input is off by a factor of ten on the second account.
- State instead of event. A buy condition that checks whether band A is below band B instead of whether it crossed below re-enters after every closed trade. The same client skeleton had this plus an iCustom call with four parameters in one place and five in another.
- Gross and net. A cycle-profit target summed DEAL_PROFIT only, without swap and commission. It closed too early, and the same trap sat one level deeper in the recovery logic.
- The four life cycles. Terminal restart, chart closed, EA removed, parameters changed. Counters in static variables make it a different EA after the first restart. Identify trades by magic number and ticket, never by comment; at least one prop-firm server rewrites comments.
Group 3: the idea does not hold
The model builds what the prompt demands. "Eleven indicators, arrow only on full consensus" is built cleanly, and the eleven votes from the same close series are correlated at 0.41 on average, effectively 3.3 independent votes. The consensus arrives a median of 16 candles after the first flip, after price has already run 2.7 ATR. And whoever lets the model build twenty variants and keeps the best one has, with high probability, selected noise.
The checklist
| # | Question, answered yes or no before money hangs on the rule |
|---|---|
| 1 | Is it written down, for every input, from when it is known? |
| 2 | Does an indicator repaint (centred, HalfLength, ZigZag, running HTF bar)? |
| 3 | Is the median gap between level and trigger under 10 minutes, and the fill at the level? |
| 4 | Does the engine find an edge on a random walk? |
| 5 | Are costs stated as a share of R per trade, and is every metric net? |
| 6 | Is the hit rate measured against stop / (stop + target)? |
| 7 | Does the test cover several regimes, walk-forward instead of one optimisation? |
| 8 | Do session boundaries come from the exchange time zone, not the server hour? |
| 9 | Are point, tick value and account currency logged, all distances in index points or R? |
| 10 | Is every entry condition an event, and was every gate observed before the first trade? |
| 11 | Is it decided at every money sum whether swap and commission belong in? |
| 12 | Were restart, chart close, EA removal and parameter change each triggered once? |
Nothing here argues against language models. They do in minutes the part that used to cost days. The expensive part was never the code. It was this list, and it comes from backtests that went wrong and accounts that reacted differently from the tester.
Full text with every case, the linked measurements behind each number and the PDF download: Building a trading strategy with AI: the mistakes you cannot see in the code
Historical statistics are no guarantee of future market behaviour. This is not investment advice.
