Fairly compare 2 different EAs. How about 1000 different EAs?

[Oleksandr Medviediev]

“Vast sea of EAs, zillion good-looking robots, each promising riches, but leaving traders and investors stranded on the shores of uncertainty.” ©

My contribution toward:
 – MetaQuotes analytical tools and metrics for evaluating strategy performance  

 – more transparent and informed trading environment  

Thinking to create something useful to last - a universal EA score/ranking system (not just XLS table), establishing standardized criteria and benchmarks for evaluating quality across all EA types, sizes and risk levels - going far beyond subjective user reviews and offering a fact-based, structured approach to EA quality.

With so many algos out there - each with flashy promises, some with impressive balance-curve visuals, many with decent track records - there’s still no uniform, recognized or reliable method for comparison and quality evaluation (beyond bug-free code), leaving traders and investors stranded on the shores of uncertainty…

STAGE 1: EA performance rating

Assign weights to key quality criteria:

 – Returns vs. risk (how much you earn for the risk you take) – 30%  

 – Drawdown and recovery (how bad it gets and how fast it bounces back) – 30%  

 – Forward-test time (live on real, not backtests - 3 months minimum, 3+ years ideal) – 20%  

 – Data stability (does it perform similarly across broker-servers) – 20%  
 - Additional metrics – optional menu: trade frequency, peak-to-valley DD, SQN, MTM, etc.

STAGE 2: Calculate final score (0–100)

Each metric is normalized, multiplied by its weight and summed up.

STAGE 3: EA ranking tiers - 4 clear groups:

 – ⭐️ Tier-1 Outstanding: 80–100 – high performance, steady 5-7 trades/day, durable and stable across all market conditions and behaviors. Price range: $800–$3000+ (up to a million if you strike gold)    

 – Good: 60–79 – solid strategy, but equity can get bumpy; 1–2 losing months per year are possible. Price range: $300–$799    

 – Mediocre: 40–59 – unremarkable, possibly overfit or inconsistent. Price range: $100–$299    

 – Poor: <40 – unstable, high risk, unreliable. Price range: $0–$99 or best avoided

The algo-trading community needs a simple, reliable Quality Score that is:  

 – objective, consistent and easy to verify  

 – globally applicable (low- mid-frequency, FX spot, retail level)

Maybe MetaQuotes builds it into MT5. Can start with ⭐️ (like Michelin) 
Could borrow from ISO or other software quality models.

Such a system could offer the clarity and transparency needed for traders and investors to choose EA with more confidence - understanding which EA is genuinely better suited for specific profitability and risk targets.

Help developers recognize their EA’s true value and fair price.  

This could naturally attract more MT5 users.

[Carl Schreiber]

I would add the behaviour in:

1. up trend, 
2. down trend,
3. flat markets,
   
4. trend reversal up -> down,
5. trend reversal down -> up,
6. change from flat to up trend,
7. change from flat to down trend.

A symbol, a time frame and a time period must be specified for each category and of course the same setup of the EA.

[Oleksandr Medviediev]

Nice zoom in - could definitely make a solid Layer-2 later on. First need to move from basic to more structured.

I was aiming to start with a standardised, system-agnostic scoring framework - kind of like a credit rating agency for EAs. Something universal that doesn’t depend on which market phase or which pair/TF you’re focused on.

[Oleksandr Medviediev]

One says: “Sharpe is our North Star”!
Another: “Sharpe is the least important”!

Both are right - it depends on goals and context.

That’s where we need MetaQuotes to step in. Not to dictate, but to standardize. Just like credit ratings - they don’t tell you how to invest, they give a consistent benchmark - that’s what we need for EAs.

[Ryan L Johnson]

Oleksandr Medviediev:
– Forward-test time (3 months plenty vs. 3 years not enough - TBD)...

I have a self-imposed minimum of 10 years of test time for my own EA's.

Knowing what we know about the Market, I wonder how many EA's would even survive in the Market all of the way through testing. I mean, a large part of the testing could be rendered obsolete by self-destructing EA's.

[Oleksandr Medviediev]

Coincidently I backtest my EAs on 10-yrs data too. Unfortunately backtests have little value for investors/potential buyers (little to none).

Real trading live - that's what truly matters for quality assessment. 

If that was easy - everyone would have ⭐️ 

[Ryan L Johnson]

Upon further brainstorming, maybe a list of the EA's that have been in the Market for 10 years or more with verified positive reviews (if possible) would be self-proving.🤔

Based on my calculations, there are presently 6544 EA's for MT5 listed in the Market, and each listing displays the initial publication date. I might manually work on this in my free time.

[Oleksandr Medviediev]

Longevity speaks volumes. A list like that would be gold!

My biggest unknown: is MQ even seeing any of this.

[Miguel Angel Vico Alba]

Oleksandr Medviediev #: My biggest unknown: is MQ even seeing any of this.

Hi Oleksandr,

I really liked the concept you proposed. It’s well thought out, and having a structured scoring model for EAs would definitely bring more clarity and fairness to the Market. Many users would appreciate a more objective way to evaluate and compare systems beyond just flashy equity curves or marketing claims.

That said, based on my experience moderating the forums and being close to how things usually work at MetaQuotes, I believe it’s unlikely that this kind of system would be officially implemented, at least not for now. While there are always administrators and developers reading the forum, they usually focus on specific areas and don't have the authority to drive large structural changes unless there’s a critical issue involved.

Also, this idea would require quite a bit of effort in terms of data collection, analytics, maintenance and transparency controls. That’s a big commitment, and unless it aligns with a broader company goal, it’s hard to see it being prioritized.

Still, I think the discussion is valuable on its own. Even if it doesn’t turn into an official feature, your scoring model could still inspire third-party initiatives or at least help raise awareness about how to judge the quality and reliability of an EA more seriously.

Thanks for sharing such a thoughtful proposal. 😉

Edit: If you really want to put this proposal on MetaQuotes' radar, I recommend opening a service desk ticket. They have a system in place to collect user suggestions and ideas, and who knows, maybe when they’re reviewing proposals, yours will come up.

[Lorentzos Roussos]

Also track usage time and display it and weigh it.

For instance a review on a product can be like this :

Product price $1399 2 copies left 

Bob Chesterfield , 5 stars , bought for $30 usage time 2 seconds Demo account

I really like this ea its profitable since week 1 and i've already made my money back , please send me TG invite

[Oleksandr Medviediev]

Miguel Angel Vico Alba #:
Also, this idea would require quite a bit of effort in terms of data collection, analytics, maintenance and transparency controls. That’s a big commitment, and unless it aligns with a broader company goal, it’s hard to see it being prioritized.

Thanks, Miguel - really appreciate your thoughtful reply. I get the constraints, but as you said, the potential is massive. Years ago, a US equity trader pitched me his robot: complex setup, mediocre (actually extremely poor) performance - but with 20-year live track record! His asking price was $11 million.

That kind of reality really makes you think. It’s serious money-making potential here - for developers, users and MQ alike. Yes, it takes effort, but the upside is enormous.