Scalping system shows zero directional edge across 31 features — is there anything left to test?

[Mohamed Trading]

I run an LLM-driven intraday scalper on MT5 (demo). Over 60 closed trades it

lost $212, and commission alone was 40% of that loss. Before asking "how do I

make it profitable", I spent two weeks trying to establish whether it has any

directional edge at all. It does not — and I want a sanity check from people

who have done this longer than me.

What I tested, and how:

- Split every dataset into a design half and a held-out test half by time.

- Built a random-entry null for every test, so any result is compared against

  coin-flip entries with the same trade count and holding time.

- Applied Benjamini-Hochberg correction (q <= 0.10) across the whole grid,

  not per-test p-values.

- Ran a positive control first: volatility clustering. It was detected at

  rho 0.30-0.84 while random controls stayed at or below 0.105 — so the

  tooling can detect a real effect when one exists.

Results:

- 31 non-price features (session, spread, tick volume, time-of-day, regime

  labels, etc.): 0 survivors.

- 13 level-based features (prior day high/low, opening range, round numbers,

  swing points): 0 survivors — hit rate was exactly the random rate.

- 48 target/stop/hold-time combinations: all negative after cost.

- 25 gate/threshold settings: none supported.

- Direction carried no information at any horizon from 5 minutes to 20 days,

  tested across 7-21 years of data depending on the symbol.

What I already fixed along the way: five classifier inputs were pinned to

constants, the cost table was wrong on three pairs, and two features had

look-ahead. All corrected before the final tests.

My questions:

1. At 40% of losses going to commission, is scalping with a ~10-15 pip stop

   structurally dead for a retail account, regardless of signal quality?

2. Is "no feature survives a holdout + null + BH correction" the normal

   outcome for this kind of search, or does it indicate a methodology error?

3. Should I be testing entry signals at all, or is trade management (exits,

   sizing, scaling) where retail edge actually lives?

4. Are there published intraday strategies with out-of-sample evidence that

   you would consider a fair starting point rather than a curve fit?

5. What sample size do you consider the minimum before a live/demo record

   means anything?

6. If you had this result, would you continue, change timeframe, or stop?

[Ryan L Johnson]

Mohamed Trading:

At 40% of losses going to commission, is scalping with a ~10-15 pip stop

   structurally dead for a retail account, regardless of signal quality?

I'm inclined to say yes, albeit with a partial caveat. The idea behind proportional commissions is get lower variable spreads, and thereby save on total transaction costs in the end. If your broker-dealer has stacked the deck in a way that hits you with higher total transaction costs, that is self-defeating. Therefore, issue #1 is your specific broker-dealer's fees. I also see that you referenced "tick volume" in your OP. Therefore, you're likely trading with an OTC FX/CFD broker-dealer. While retail OTC broker-dealers are rather notorious for having high commissions and spreads, centralized exchanges/real futures broker-dealers are not. For example with one CME futures broker-dealer, the average spread on 1OZ (nano) gold futures is 25 points (0.25 USD) plus a 1.54 USD commission per round trade (discounts for higher trade volume, and MGC (micro) and GC (full) have proportionally cheaper fees. Additionally, real contract volume is available. Therefore, issue #2 is the market in which you're participating.

In any case, you are still a retail trader having a "retail account"─not a professional trader. Therefore, the aforementioned caveat is the fact that all types of retail trading are not equal.

[Ayaz]

On sample size, a number off a real account rather than a rule of thumb.

I went through 801 closed deals on one account for a report. At its own stop and target the strategy needed an 88.50% win rate to break even and it got 94.13%. That reads like a comfortable margin. In deals it is 45 out of 801, so everything that account ever earned sat inside 45 trades.

Put your 60 against that. The same margin over 60 deals is about three trades, and one standard deviation on 60 coin flips is close to four. So a 60 trade record cannot tell you whether direction carries information, even if it does. That is a separate question from your feature work, which is far better powered.

On the settings, before accepting that none are supported: sort the trade list from each one and hash it. I had four configurations differing by a single input that produced two results, not four, identical byte for byte, the summary lines differing only by rounding. If some of yours collapse the same way, your grid is smaller than you think and the correction was stricter than it needed to be.

[Mohamed Trading]

ريان لدوار جديد :

أميل إلى الموافقة، مع بعض التحفظات. وضع فكرة المعاملات النسبية في تقليل فروق الأسعار المتغيرة، وبالتالي توفير تكاليف المعاملات في النهاية. إذا افترضت أن وسيطك قد توصل إلى اتفاقك، فهذا يعني أنه سيصلحك. المشكلة، المشكلة الأولى في تحليل وسيطك. وألاحظ أيضًا أنشر إلى "حجم اللعب " في منشورك الأصلي. لذلك، من خلال فهمك للتداول باستخدام تداول العملات الأجنبية/عقود التداول خارج البورصة. في حين أن وسطاء التداول يعتبرون التداول معروفون بمولاتهم وأرقامهم، وإن وسطاء البورصات المركزية/العقود الآجلة رمزها كذلك. على سبيل المثال، مع أحد وسطاء العقود التجارية في بورصة شيكاغو، لفرق متوسطة للعقود الذهبية الآجلة لوزن أونصة واحدة (نانو) 25 نقطة (0.25 دولار أمريكي) بالإضافة إلى عمولة تشمل 1.54 دولار أمريكي لكل جولة (مع وجود خصومات على أحجام اللعب الكبيرة ، ورسوم أقل للعقود MGC (مايكرو) وGC (فول)). علاوة على ذلك، التخصص الشائع للعقود. لذا، الثالثة الثانية هي السوق الذي تشارك فيه.

على أي حال، أنت لا تزال متداولًا فرديًا بحساب تداول فردي، وليس متداولًا محترفًا. لذا، انتبهوا إلى أنفًا هو جميع أنواع الأشخاص ليسوا أول من يفعل ذلك.

Thank you — you identified the venue from one word, and you were right.

We had measured the same thing independently before your reply. Splitting

our broker's overnight financing against published central-bank policy

rates: about 85% of it is markup, not market carry. And across 150

published swap sides on 75 instruments, not one pays the side that a

rate differential says should receive. So the charge is taken in both

directions on every instrument.

Your CME figures give me the comparison I was missing, so two questions:

1. Which broker are those nano/micro gold figures from? I'd like to check

   the full fee schedule rather than work from an average.

2. Before I build anything on this: are US futures accounts generally

   accessible outside the US, and are there minimum deposits that make

   micro contracts impractical for a small account? I'd rather know the

   constraint now than after the work.

The real contract volume point is separate and I hadn't weighed it. Every

volume feature we tested was broker tick volume, which we now treat as a

broker artifact rather than market data. If real volume is available, that

is a data source we have never actually had.

[Mohamed Trading]

Ayaz #:

On sample size, a number off a real account rather than a rule of thumb.

I went through 801 closed deals on one account for a report. At its own stop and target the strategy needed an 88.50% win rate to break even and it got 94.13%. That reads like a comfortable margin. In deals it is 45 out of 801, so everything that account ever earned sat inside 45 trades.

Put your 60 against that. The same margin over 60 deals is about three trades, and one standard deviation on 60 coin flips is close to four. So a 60 trade record cannot tell you whether direction carries information, even if it does. That is a separate question from your feature work, which is far better powered.

On the settings, before accepting that none are supported: sort the trade list from each one and hash it. I had four configurations differing by a single input that produced two results, not four, identical byte for byte, the summary lines differing only by rounding. If some of yours collapse the same way, your grid is smaller than you think and the correction was stricter than it needed to be.

This is the most useful reply I have received, on both points.

On sample size: your 801-deal comparison settles it, and it matches an

internal audit we ran independently. Our 60-trade record gives 47.5%,

with a 35.3%-60.0% confidence interval and p=0.54 — it distinguishes

nothing. Your framing of it in trades rather than percentages is what

made it land: 3 trades of margin against 4 of noise. And your point that

this is a separate question from the feature work is exactly the

distinction I had been blurring.

On the grid: I am running your hash check now, across every family we

corrected — gate settings, target/stop/hold, horizons, levels, pairs,

positioning, calendar. We already had one instance of exactly what you

describe and never generalised it: 20 selected pairs that reduced to

about six distinct spreads. If others collapse the same way, our

corrections were stricter than they needed to be. I will post the result

here either way.

One question about your own record, if you don't mind. A strategy needing

88.50% to break even and returning 94.13% implies a very small target

against a large stop — many small wins and rare large losses. That is a

completely different structure from what we built (roughly 1:1.3, aiming

for a directional edge). Was that structure a deliberate choice, or what

survived testing? I ask because our whole search assumed the edge had to

come from direction, and your numbers suggest the structure itself may

carry more than the signal does.

[Arnold Holm]

Your null result is a useful result. I would not try to rescue the same trade list by searching more exits and position sizes. First check the test plumbing with one timestamped fixture: signal time, entry and exit prices, spread, commission, broker time, and the log line showing when the signal existed. You already found pinned inputs and look-ahead, so that audit matters.

Then freeze one new hypothesis before opening another holdout: signal, costs, maximum number of variants, and the condition that ends the search. Compare it with a matched random-entry null before choosing stops or sizing. An exit rule may change the distribution, but that is a new hypothesis to test, not evidence that the original entry carried direction.

If the corrected pipeline still shows no edge after costs, I would pause this scalper. Another venue or longer timeframe would be a fresh test with its own untouched period. The 60 demo trades help check execution; they cannot settle whether the strategy has an edge. There is no universal minimum trade count. Define the smallest improvement that would clear your real costs first, then size the test around that.

[Ayaz]

Mohamed Trading #:

One question about your own record, if you don't mind. A strategy needing

88.50% to break even and returning 94.13% implies a very small target

against a large stop — many small wins and rare large losses. That is a

completely different structure from what we built (roughly 1:1.3, aiming

for a directional edge). Was that structure a deliberate choice, or what

survived testing? I ask because our whole search assumed the edge had to

come from direction, and your numbers suggest the structure itself may

carry more than the signal does.

Not a choice of mine. That was a client's EA. Much of my work is reports on EAs people already run: from the trade history, what the EA needs to win just to break even, what it actually wins, and how many trades separate the two.

That's also why I'd be careful reading edge into the shape. It doesn't carry edge, it moves the bar. Another EA I reported on had the same shape over a sixteen-year backtest: average win 4.80 against average loss 19.14, so it needed about 80% to break even. It won 74% and the test account went to zero. Same structure, opposite result. What decided both was the win rate against its own break-even line.

It also makes a short sample even harder to read. At 88.50% break-even one loss is worth nearly eight wins, so over 60 trades a handful of losses decides everything.

Curious what the hash check turns up.

[Mohamed Trading]

Ayaz #:

Not a choice of mine. That was a client's EA. Much of my work is reports on EAs people already run: from the trade history, what the EA needs to win just to break even, what it actually wins, and how many trades separate the two.

That's also why I'd be careful reading edge into the shape. It doesn't carry edge, it moves the bar. Another EA I reported on had the same shape over a sixteen-year backtest: average win 4.80 against average loss 19.14, so it needed about 80% to break even. It won 74% and the test account went to zero. Same structure, opposite result. What decided both was the win rate against its own break-even line.

It also makes a short sample even harder to read. At 88.50% break-even one loss is worth nearly eight wins, so over 60 trades a handful of losses decides everything.

Curious what the hash check turns up.

Ran your hash check across every family we corrected. You were right in

principle, and here is what it found.

Two families were over-counted: the target/stop/hold grid, 48 declared

and 42 distinct, and the gate grid, 25 declared and 20 distinct. The

positioning grid by one cell. Five other studies: no collapse at all.

No BH verdict changes anywhere. The corrections were stricter than they

needed to be, but no p-value was close enough for it to matter — the

smallest in the two affected families were 0.157 and 0.204 against

thresholds near 0.002.

The more useful thing it exposed was structural. Our "clean trend" gate

had five threshold settings; all five produce one identical list of 586

trades. The line relabels routes, it does not gate them. We had

suspected that from two other directions and this settled it. It is

being removed.

Two limits on the method, in case they are useful to you: the hash

catches identical cells but not near-copies, which are correlated

without matching byte for byte, so independence is still lower than the

corrected count suggests. And identical cells carried recorded p-values

up to 0.15 apart purely from the null's random draws — a cell near a

threshold could flip on the seed alone.

And thank you for the correction on shape. I had started to read edge

into it and you shut that down with a counter-example, which is worth

more than agreement. "It doesn't carry edge, it moves the bar" is going

into our own notes. Your 4.80 against 19.14 case is the clearest

argument against the reasoning I was about to follow.

Your line of work makes sense of the earlier reply too. Reading trade

history for what a system needs to win against what it actually wins is

exactly the measurement we lacked for months.

[Ayaz]

Your first post already has what you'd need for that measurement. If I've read 1:1.3 right, target 1.3 times the stop, the bar before costs is about 43.5%. You won 47.5%, so on paper the 60 trades cleared it.

But commission was 40% of the $212, which leaves about $127 lost before commission. At 47.5% wins that doesn't happen unless the average win came out under 1.11 times the average loss. So what the trades delivered was nearer 1:1.1 than 1:1.3, and at that ratio the bar is above 47.5% before commission is even counted.

On the p-value spread, seed each cell's null from its trade-list hash. Copies then get the same p-value every time. A cell near a threshold still needs more draws, but the copies stop disagreeing.

Do all your exits go through the stop and target, or does the model close some trades itself?

[Mohamed Trading]

Ayaz #:

Your first post already has what you'd need for that measurement. If I've read 1:1.3 right, target 1.3 times the stop, the bar before costs is about 43.5%. You won 47.5%, so on paper the 60 trades cleared it.

But commission was 40% of the $212, which leaves about $127 lost before commission. At 47.5% wins that doesn't happen unless the average win came out under 1.11 times the average loss. So what the trades delivered was nearer 1:1.1 than 1:1.3, and at that ratio the bar is above 47.5% before commission is even counted.

On the p-value spread, seed each cell's null from its trade-list hash. Copies then get the same p-value every time. A cell near a threshold still needs more draws, but the copies stop disagreeing.

Do all your exits go through the stop and target, or does the model close some trades itself?

Your arithmetic is right, and our own numbers make it worse.

Recorded average win 0.882 R against average loss 0.973 R, so the

realised ratio was 0.906 — not the 1:1.3 the trades were designed for,

and below 1:1 outright. The design bar was 43.5%; the delivered bar was

above what we won.

What stands out is that the average WIN came in under 1 R while the

target sat at 1.3 R. Something is closing winners early.

To your question: I don't yet know, and I am not going to guess. It is

now the first thing being checked. There is a model in the loop making

the entry decision, and whether it also closes positions itself, or

whether every exit goes through stop and target, is exactly what I am

asking the codebase. I will post the answer here either way.

If it turns out the model closes trades itself, then our whole search

was aimed at the wrong half of the problem. We spent weeks testing 5,700

cells of entry features for direction, and none of that would matter if

the exit was giving back the geometry the entry was sized for. Your

question may have found in one line what we did not find in weeks.

On the seeds: adopted. Seeding each cell's null from its trade-list hash

is the right fix — copies stop disagreeing by construction, and the

near-copies the hash cannot see at least stop adding noise on top. Going

into our test tooling as a rule.

[Timon-pascal Krueger]

Arnold's point about exits matches what I measured. I ran 18,900 trade management policies on a surrogate market, identical code on a series built so that no pattern can exist, and all of them ended at minus the costs. Exits change the shape of the results, win rate against average win, but on entries without information they do not create expectancy.

On your question 4, I rebuilt the Zarattini opening range breakout on Nasdaq rule for rule and got the paper back gross, +0.13 R per trade at t 2.9. With CFD costs it came out at +0.002 R, and negative on the four other indices I tried.

On question 2, I recently scanned 15 markets and took 16 candidates that survived discovery into a separate holdout from 2023 on. None held after a correction across the 16, and every discovery t between 2.2 and 4.9 shrank. My best real finding was t 3.65 while the best finding on random data had a median of 3.81. So finding nothing is the normal result, and your nulls are the reason you noticed.