Scalping system shows zero directional edge across 31 features — is there anything left to test? - page 3

New comment
Timon-pascal Krueger #:

Mohamed, it starts from the real minute bars of the same market. Each bar is mirrored around its open with probability one half and every gap between two bars gets a random sign, all in log space so prices stay positive. The sign is a fresh draw for each minute and has nothing to do with what came before, so the next bar's direction is a coin flip by construction and no rule built on past prices can predict it, however many variants you search. What stays real is the size of every bar, so volatility clustering, the intraday volatility profile and the distribution of ranges are the ones your stops and trails meet in the real market. That is why management can be tested on it at all. As a check I ran 379 tests on the series and 15 came out beyond t 2, where chance alone gives about 17. One limit: because the magnitudes are kept, it only rules out directional patterns, so a rule that trades volatility itself is not tested by it.

Timon — this is directly useful to us, so let me be specific.
Our null permutes returns. Against a declared 10% false-alarm rate it fires at 22.2%, and 49.0% in a stricter world that resamples whole broker-days. Cause: permutation spreads volatility evenly, so candidates that trade volatile bars get an inflated statistic. Your series keeps each magnitude at its own timestamp, which removes that mechanism. We will implement it and post our own false-alarm and detection numbers here, at 10,000 zero worlds and 1,000 planted — positive or negative.

Two questions before we do.
1. Mirroring a bar around its open swaps its upper and lower wick, so across the series any up/down wick asymmetry is symmetrised. Range and clustering survive; the side a wick reaches does not. Since a long's stop is hit by the lower wick and a short's by the upper one, that bears on the use you single out — testing management. Have you compared stop-hit rates against the real series, by side?
2. In our data the bars inside a trade carry about 9% more variance than independence implies, and none of our six nulls models it. Because your method keeps every magnitude in place, it may preserve that dependence. Do you know whether it does?
With 60 trades you cannot separate skill from luck: the standard error of your mean trade result is the trade standard deviation divided by sqrt(60), so unless the average trade is several times that, the confidence interval will comfortably include zero. Compute it directly (mean, standard deviation, 95% interval) on net results after commission and swap, and also run a random-entry baseline with the same exits and costs to see where your result falls in that distribution. Your own numbers already point at costs: if commission is ~40% of the loss, a 10-15 pip stop with a tight target needs an unrealistically high hit rate to cover spread plus commission, so measure cost per trade in R before testing more entry features. As a rough rule, plan for a few hundred trades before judging an edge, and treat anything you tuned on as in-sample. If the edge is zero before costs, changing exits or sizing will not create one; test exits only if the pre-cost expectancy is positive.
Abdullah Uygar Tuna #:
With 60 trades you cannot separate skill from luck: the standard error of your mean trade result is the trade standard deviation divided by sqrt(60), so unless the average trade is several times that, the confidence interval will comfortably include zero. Compute it directly (mean, standard deviation, 95% interval) on net results after commission and swap, and also run a random-entry baseline with the same exits and costs to see where your result falls in that distribution. Your own numbers already point at costs: if commission is ~40% of the loss, a 10-15 pip stop with a tight target needs an unrealistically high hit rate to cover spread plus commission, so measure cost per trade in R before testing more entry features. As a rough rule, plan for a few hundred trades before judging an edge, and treat anything you tuned on as in-sample. If the edge is zero before costs, changing exits or sizing will not create one; test exits only if the pre-cost expectancy is positive.

I promised to post this number either way, so here it is.

Pre-cost: −0.036 R a trade (95 %: −0.298 to +0.227). Post-cost: −0.118 R. Measured cost 0.0824 R — commission 0.060, the spread at each trade’s own minutes 0.023, slippage about zero.

Three things, and the third is why I’m posting rather than just thanking you.

1. Cost is 70 % of the loss — and inside it, commission is 73 %, not spread. Every lever we were weighing (timeframe, instrument, session) moves the spread. The bigger half is a line in a broker agreement we had never looked at.

2. Slippage is about zero. We expected it to be hiding and to pull pre-cost toward zero. It didn’t: −0.038 estimated, −0.036 measured. Execution quality isn’t our problem.

3. Your criterion is correct and unrunnable here. Holding the observed SD (1.03 R), the interval excludes zero at about 3,265 trades. We produce 64–72 entries a year — 45 to 51 years. So the live book isn’t an instrument of measurement at all: 60 draws spanning −0.38 to +0.14 are consistent with a small edge, with no edge, and with a small negative edge at the same time. We had been reading it as evidence. It isn’t, either way.

Per instrument: XAUUSD 24, NAS100 19, US30 16, EURUSD 1 — every cell under 25, so no cell is a verdict, including the positive one (US30 +0.268, interval −0.33 to +0.86).

Two honest limits: these 60 predate a fix to our classifier inputs, and 11 of them still have an estimated rather than a measured spread.

What it changes for us: the decisive work has to come from historical testing, not the live account — and cost moves ahead of entries, exactly as you put it.

One question back: when cost is fixed by the broker and the minimum detectable effect sits above it, is there entry research you’d still consider worth doing — or is that the point where the instrument itself is the wrong instrument?

123
New comment