Fairly compare 2 different EAs. How about 1000 different EAs? - page 5

 
Lorentzos Roussos #:

It would take sellers creating profitable EAs and selling them at a high price to dominate the first page 

They'd win -> clients would win -> metaquotes would win

(also if there were any holes in the market upload process , those getting dripped from the holes would also benefit , hypothetically)

Exactly this.
 
Vladimir #:

You’re conflating two different mechanisms. My original point was about systematic inefficiencies - like arbitrage or statistical edges - that erode as more participants exploit them (e.g., high-frequency traders crowding out a price discrepancy). This is a well-documented market phenomenon.

The GameStop saga was fundamentally different: It was a liquidity squeeze fueled by collective retail action (and institutional short positions), not an algorithmic edge being arbitraged away. The institutional backlash wasn’t about inefficiency decay - it was about losing a zero-sum bet while retail traders exploited a structural vulnerability (short interest >100%).

You're making a lot of false assumptions about the types of EA's that traders would be selling. Given that an arbitrage or HFT EA would get your forex/CFD account terminated in short order, I really don't see how it's relevant. Furthermore, profitable swing and positional trading EA's are ultimately based on a statistical edge--or else they would cease to be profitable.

Regarding GameStop, you're trying to present the motivation of the GameStop traders as if it were a mechanism. The actual mechanism is quite simple--buyers versus sellers which is a universal truth in swing/positional trading.

 
Lorentzos Roussos #:

It would take sellers creating profitable EAs and selling them at a high price to dominate the first page 

They'd win -> clients would win -> metaquotes would win

(also if there were any holes in the market upload process , those getting dripped from the holes would also benefit , hypothetically)

Oleksandr Medviediev #:
Exactly this.

As if anyone really needed data to prove this... I'm about 800 EA's deep into my list and the Market's priority is beyond confirmed.

 

Hi to all! I usually don't participate in mql5's forum discussions as most of the time they are poor quality or out of the topic. But I would gladly to jump to this one.

I think the proper ranking and understanding of EA's performance is a very serious problem, the debates about "why would anyone sell something etc...." or "why someone need to sell its product etc..." , maybe this can be sociological aspect to study, but from the technical point of view will just add more confusion and more people will come here who don't have a clue about what technical aspects and what implications are involved. And if we mix these 2 things we will end with a lot of theories and probably non sense that will not add any value to this topic. Maybe if the people are really interested in a such conversations where anyone can say its personal view on how this market work, maybe it would be better open another thread for that, where anyone can express its personal theory based on nothing or based on partial knowledge of facts or anything else.

Said that, let's back to the technical aspects. Before analyzing profitability of the EAs, their reliability or resilience, we should check for their correctness, in every aspect of it, and with correctness I mean we should check that the EA is not cheating, on purpose by the author or by mistake. This is a very big problem here because we are not allowed to discuss the product publicly. Speaking about cheating, the metaquotes ranking system is involved too, I think we all know that, but this certainly is not helping and bogus EAs proliferate.


What you call longevity of the EA is not by any means guarantee of its goodness, even with the live signal. Let's forget for a moment that there are bogus broker that will provide you a master account to alter the history of trades(for a relative low price), let's forget that case for a moment. These live signals are traded with different trading system during years(probably similar but different), so what you see it is not belonging to the EA you are buying/renting. Also they exploit the fact that we can't download previous version of the EA(mql5 site doesn't allow that) so they keep updating them to make the past look great in the backtester.

A very common example of this is a classical poor martingale system that, let' say, works/is profitable most of the time, but then it blown the account. The author will add extra funds to his live signal, for example on 1000usd he will add another 3000usd, the system will work out and then it will withdraw its 3000usd to use them for the next time. The way the live signal stats are mode in this site will show low DD and at first sight the live system will still look good. After that he will immediately release a new version of its EA, stating there is a bug fix or something, but practically he will just upload a new version of the EA with a hardcoded rule so in the backtest the EA will skip that particular trade sequence that would lead to blown the account. And voilà, a small magic is done. New buyers will see the long live signal alive, the backtest will look wonderful.


I was paid in the past by customer to actually check one of these "TOP" seller EA(using the demo version from mql5) and the results were terrible. I found so much cheating that you have no idea. One of these EA that puts buy and sell stops above a certain price range(defined by time, so the min and max within these candle), I can't say the name but you certainly can spot out which one is. The EA had very clear params where you could define how many points you wanted to put this stops above and bellow. Well in the days that the EA would start the sequence that would lead to blow the account, these params were not never respected and they were put so much higher to avoid the price to trigger them. An this in many days, the funny part is if you run that on another instruments, the params are respected in all days but not in the one hardcoded(that are obviously the same in all pairs). This is a clear proof they were hardcoded, but also in a not so clever way. This is the easiest cheat I can bring as example, but there were many more, I found approximately 50 altered trades and days in the last 10 years(with many different ways but with mainly 3 methodology).

I was really surprised that no one actually noticed this, in youtube I found a huge amount of review of this particular system with a lot of people who explain his personal theories about this and that and probably have no clue whet they are speaking about. It is just show entertainment. I repeat, this case is just an example that was easy to explain in the forum, but there are plenty.


Then for the amusing of the readers, there is even a worse part of the story. Because of course, if a martingale system blown the account from time to time, that should be reflected in the reviews. Well, I have personally contacted a couple of guys when I saw their bad review and they were were angry as felt scammed, but all of them removed their review because of the author offered them a refund of the paid price-10%, that is a lot of money for these TOP seller EAs. Another tecnique is to force the buyer to leave a good review(after a week) in order to get new tools or "special" set file that "work" better.

I mean, let's be honest, we all know that this kind of things happens here, this is a very big problem.

Said that, I would really like to help you to make an algo to rank EA or at least to estimate its good and bad points, but with the evidence based techniques and not as a usual youtuber reviewer that has zero, repeat zero understanding of what he is saying and proposes nonsense. I think that any tools or approach used should be scientifically tested and we can't assume it is not bogus just because it is "used" by the industry(sell by the industry), we have to do our own homework, and I know this will take a lot of time, but once we will get rid of all garbage what remain will be actually usable in trading. I would remind you all, that there is a special broker out there that estimates the risk involved by your EA using a proprietary VaR(it uses that proprietary VaR as a legal contract as an official Asset Manager), that is a very effective measure for the big investor, for the details I suggest to google it or drop me a message and I will send you to the page of explanation. (I wonder what would be the VaR of these EA listed in mql5's site).

Sorry for the long thread, didn't know how to start, I wanted to put a lot more info on this, but it is already too long, I will wait to hear from you first, before I add more infos.

Regards


EDIT:

I forget to say one very important thing. If we want to check these EA(for the cheating), there is a practical way to do so, just collect all the versions during a year and after that use them to run in backtester, that would be a run in the future for them on completely unseen data. The MT5 backtester is really good if used properly especially in the non seen data. So making a DB with all these EA would be really helpful, the issue is we should choose one machine to do so, as every version is bond to a specific machine.

 

Deep-dive, Aleksandar! Brings real weight to the discussion. Some parts were eye-opening for me personally too. Curious about that VaR-based risk evaluation you mentioned. Worth discussing here (unpack openly).

My biggest unknown is MQ’s stance on all this - do they know? Do they care? Will we see any change soon? Anyone have insight?

 
Oleksandr Medviediev #:
My biggest unknown is MQ’s stance on all this - do they know? Do they care? Will we see any change soon? Anyone have insight?

Example.

Any scam should be technically proven. If the user do not have any technical proof or do not care about it so - he/user is the scammer.
It is the rules of trolling.

How the service desk is dealing with the scammers? they are deleteting their profiles completely from the forum ... irrespective off - he is seller so his profile will be deleted together with his EAs published ... or just simple user who told something (because one of them is scammer just for sure).

It is joking example but it may be the true sorry.

----------------

So, proof, proof and one more proof.
To the service desk of course.

----------------

We (the forum users together with me and other forum moderators) do not moderate the Market. We can make just the proposal for possible improvements.

 

The other situation ...

We know that one EA on one broker and same EA with the other broker can not be compared with each other ... because the quotes/data is different for every broker. EA may be profitable for several years for one person (with one broker/data or time to start trading) and completely losing with the other broker/data or time to start trading.

So, how to compare two different EAs by performance if it is impossible even if for one same EA ...

----------------

The titile of the thread is the following:
"Fairly compare 2 different EAs. How about 1000 different EAs?"

So, I may propose to change the title to be like that -
"Fairly compare 1 same EA with different brokers and time to start trading. How about 2 different EAs?"
:)
 
Aleksandar Petrinic #:

Said that, I would really like to help you to make an algo to rank EA or at least to estimate its good and bad points, but with the evidence based techniques and not as a usual youtuber reviewer that has zero, repeat zero understanding of what he is saying and proposes nonsense. I think that any tools or approach used should be scientifically tested and we can't assume it is not bogus just because it is "used" by the industry(sell by the industry), we have to do our own homework, and I know this will take a lot of time, but once we will get rid of all garbage what remain will be actually usable in trading.

At the link (autotranslate) I offered a ready-made implementation of the algorithm (open source code), which allows you to immediately evaluate the history of trading for increased risks. That is, when a beautiful yield curve depends more on MM than on the quality of trading signals.
Нестандартный анализ истории торговли.
Нестандартный анализ истории торговли.
  • www.mql5.com
После того, как ТС прошла массу проверок на бэктестах/демо, приходит время реальной торговли. Эта логика порождена двумя гипотезами: Торговля на реальном счете и затем прогон на истории покажут
 

"poor products" ... means: there are bad EAs and good EAs ... based on the performance (backtesting or trading)  ...
But it is impossible to do because every single same EA may be profitale or losing depends on the broker and time to start trading ...
So, the performance of the EA can not be the main criteria to be good or bad product ...
--------------------

I know one/main/key criteria only: who is selling (and how long we all know this person). Or bad coding or good coding (it is also related to the person who is selling).
We are investing to the seller by buying some EA on the Market. And, of course, the buyers care about the following: who is seller, is he well-known and so on ...  But if someone is buying anything from unknown anyone just because he/someone is having a lot of money in the pocket so sorry ...
:)

--------------------

So, the main criteria for rating of the products should be the personality of the seller. Because it is some kind of believing anyway.

--------------------

If my opinion comes with the contradictions with the others - so sorry.

 
Sergey Golubev #We (the forum users together with me and other forum moderators) do not moderate the Market. We can make just the proposal for possible improvements.

Thank you, Sergey. As you rightly said, neither we as users nor even as moderators have the ability to intervene directly in the Market. The most we can do is share ideas and suggestions in the hope that they may be taken into consideration. But the final decision always lies with MetaQuotes. It is their platform, and only they can decide whether a proposal fits their vision and priorities.

In that sense, it is not particularly helpful to keep asking questions like "Is MetaQuotes aware of this? Do they care? Will they do something soon?" While understandable, this kind of reaction tends to reflect a frustration with something that is ultimately out of our control.

We also have to remember that, in the end, this is a responsibility that falls on the user, who is the first and most interested party. Expecting an absolute rating system to decide what is good or bad would effectively shift that responsibility to MetaQuotes, and no company in its right mind would take on such a burden. Officially classifying which EA is good or bad would open the door to serious legal, commercial, and technical implications.

That is why I believe the best path forward is to continue sharing thoughtful suggestions respectfully, without expecting them to be mandatory, and always understanding that the final responsibility to research, compare, and decide lies with the user.