Trading Systems: The Magic of Filtration

 

New article The Magic of Filtration has been published:

Most of the automated trading systems developers use some form of trading signals filtration. In this article, we explore the creation and implementation of bandpass and discrete filters for Expert Advisors, to improve the characteristics of the automated trading system.

Author: Sergey Pavlov

 

Great Article,

for the newbie in system developing this should be the guide to filtering...absolutely love this!

 
I'm a Newbie system developer and all the methods the author describes is exactly how I optimized my first couple of systems. I did those thing in that exact order as a matter of common sense. What I was hoping to get from this article are examples of using one indicator to filter out another. Example if (Macd<0 && CCI < 0). This approach have NEVER worked, all it does is decrease the # of Trades while killing a profitable CCI system. However, this is a very common technique in manual trading system.
 

http://pixiefx.web.fc2.com/index.html

http://www.ustream.tv/channel/fx-on-com-perrier

This is a great trading system.

 

I've turned this into a great system yielding 500% with less than 20% drawdown. Sergy's code was excellent. The hourly filter was not ideal for me however. I removed all of that code. Note-i use it on 15 min, it produces more than 5 or 1 after my filters.

 
LBranjord:

I've turned this into a great system yielding 500% with less than 20% drawdown. Sergy's code was excellent. The hourly filter was not ideal for me however. I removed all of that code. Note-i use it on 15 min, it produces more than 5 or 1 after my filters.


Love this one, have it on demo now, doing great. FXDD is the broker. who knows how long it will work for but the idea behind it is cool.

 
LBranjord:

I've turned this into a great system yielding 500% with less than 20% drawdown. Sergy's code was excellent. The hourly filter was not ideal for me however. I removed all of that code. Note-i use it on 15 min, it produces more than 5 or 1 after my filters.


I tested this but was unable to obtain anywhere near the results you mentioned. Are you using additional filtering?
 

The author of this article is abusing the terms "band-pass" and "discrete" filtering. Though what they are referring to technically reduces the number of trades one may take, and thus "filters" the trades it is not really a digital nor a discrete, nor a band-pass filter.

http://en.wikipedia.org/wiki/Filter_%28signal_processing%29

http://en.wikipedia.org/wiki/Band-pass_filter

http://en.wikipedia.org/wiki/Digital_filter

 
antisyzygy:

The author of this article is abusing the terms "band-pass" and "discrete" filtering. Though what they are referring to technically reduces the number of trades one may take, and thus "filters" the trades it is not really a digital nor a discrete, nor a band-pass filter.

http://en.wikipedia.org/wiki/Filter_%28signal_processing%29

http://en.wikipedia.org/wiki/Band-pass_filter

http://en.wikipedia.org/wiki/Digital_filter


Yes, it seems having naming issue.

When we assume:

0 = PEROID_M1
1 = PEROID_M5
2 = PEROID_M15
3 = PEROID_M30
4 = PEROID_H1
5 = PEROID_H4

We can use the example P-filter as the so called D-filter.

While not a self explanatory, but still can get the meaning in context.

If you think this article worth reading, vote it for Sergey.

http://v4ex.com/node/13

 

It is remarkable, very much helpful information for because i Have just enter in the field of system developing.........

Thanks for the effort.


sure shot mcx tips !! Mcx Tips Free Trial

 

While this filtering has improved the performance of this example somewhat, it is not really anywhere near effective enough to be a long term, solid, robust and profitable performer.

My contention and focus is in regards to entrance strategy versus exit strategy. While one must have them both to have successful, robust and profitable EAs, if the EA is not getting into a significantly higher portion of profitable trades than losing trades, this is all for naught as Sergey himself states at the beginning of this article:

If the Expert Adviser is unprofitable ("drainer"), it is unlikely that some type of filtration will help,- the magic of filtering is powerless here.

Thus I believe that first and foremost, the challenge and the task is to find the entry conditions that results in the EA entering into significantly more winning trades than losing trades to start with. If one finds and applies an effective entrance strategy that has a very high accuracy rate of getting into profitable trades in the first place, preferably above 90%, and 80% at the very least; then subsequent filtering can improve the returns and the overall robustness and result in hopefully long term profitable EAs.

As Sergy himself notes: if the filtering out of losing trades significantly reduces the number of overall trades, then it may likewise be unproductive. If their is little or almost no trading going on, then there is no reasonable possibility of it being worthwhile. I am currently doing some filtering on such a mediocre EA that the recommended setting of the StopLoss is 20 TIMES what the TakeProfit is. This means that in order to just break even, it has to have an accuracy rate of 95%! At a 45% accuracy rate, this EA is no where near this accurate by far. This to pretty well amounts to 'guessing' at when to enter into trades as opposed to sound, solid parameters based on proven profitable results. Optimization that encompassed the recommended settings 24/5 over 3+ years of data on $3K produced only $195 ~ $160 profit with DrawDowns of between 20% ~ 35%. Filtering it by time has clearly shown that only trading it from 5 to 6 and from 23 to 24 are clearly the most profitable by far. A partial Optimization on ~ 5+ years of data starting with only $1K and trading only between 5 to 6 AM returned a maximum profit of $1,117 at a DD of ~ 22% == quite high. A more reasonable DD of ~ 14.6% returned a profit of ~ $663. // ~ 11.1% DD ~ = $141. // ~ 7.6% DD ~ = $306 and a DD of only ~ 5.7% yields $266. So filtration has clearly increased the profitability of this mediocre EA. The initial 24/5 optimized results had an accuracy of only ~ 42% of the trades being profitable. The results of a partial optimization of trading at only the most profitable period for 5AM to 6 Am for this EA still has an accuracy rate of less than 50%

The problem? Optimized trading 24/5 on $3K for 3+ years gives a maximum profit of only #193.50 on $3K made ONLY between 19 - 29 trades in over 3+ years! )< 8(

The partially Optimized results filtered to only trade between the most profitable hours of 5AM to 6AM on only $1K returned a maximum profit of $1,117 but with a DD of ~ 22% // ~ $306 with ~ 7.6% DD // $266 with ~ 5.7% DD Clearly improved ROI but with about the same accuracy rate. But only 18 - 23 trades in over 5+ years! This clearly indicates that having spent $50 on this loser, and the 2 other more 'advanced' versions of this EA that were more expensive but even less profitable; that any further time invested on this is clearly wasted and would only reduce my net ROI even more! Except for the very small number of trades, these are typical results for most commercial EAs that unfortunately I have to much personal experience with )< 8) Results indicate that making more trades with this EA would make it into an even bigger loser In fact this is one of the better performing ones! )< 8(

The problem? Trading during only the most profitable period of between 5Am to 6AM, it only makes 19 trades at the most over a period of 5+ years of optimization! Hardly enough to sustain one. One might put almost all of their account onto these few trades, but without an absolutely 100% accuracy rate, this can only lead to disaster.

I have such a VERY accurate commercial EA. However it trades so rarely: 5 ~ 6 time per month on 3 charts with the recommended exposure of only 0.1% of ones account, that only one losing trade, which was by far, far larger than the winning trades, wiped our several months of profitable trading at 100 % accuracy and left it at a net loss after all this time! )< 8(

Most EAs seem to rely VERY heavily on setting the Stop Loss many, many times higher than the Take Profit with the faulty logic that it is so high that one will never hit it and therefore never lose. One of the most successful commercial EA's primary trading pair and algorithm functions this way: the recommended SL is 25 TIMES higher than the TP! In my option this is not really a valid method to the point that it can not really be called and 'Expert' Adviser at all. It is really no significant and effective method at all. This EA went along making 10 ~ 20 pips occasionally with no great regularity and then with the recommended default setting with the Stop Loss set at 500, got 'married' to one such losing trade that got as much as 450 pips in the hole and that I finally manged to get out of manually when it was 'ONLY' a 300 pip loss! )< 8( This is with what likely is the most successful commercial EA to date! Clearly the bar is set VERY low in this market! I have seen MANY such (commercial) EAs that operate on this principle of the SL being MANY TIMES higher than the TP and everyone of them that does so are all losers. IMHO these are EAs that are cranked out one after another by people that while they may be excellent MQL programmers that can produce many 'working and functional' EAs in a short periods of time and have them mass marketed; but that they have no real knowledge, skill or proven ability to produce a profitable ForEx trading system, automated or otherwise. So they keep cranking out these losers that many people jump on and utilizing mass marketing appealing not to logic and 'Proof of Performance' but are sold by utilizing hype that grabs people by there emotions and greed by indicating that they will become very wealthy in very short periods of time with them that leads them to leave all common sense behind and act on emotions and the only ones making any money off of these are those unscrupulous reprobates that produce, market and sell them.

ForEx EAs have the dubious distinction of having the highest return rate of any and all digital download products world wide! With the aforementioned losing algorithms making up most of the EAs, and the very poor ROI, which is negative in most cases, on virtually all of them; it is no wonder. Like InterNet marketing products themselves, there are so many that utilize so many faulty and baseless claims that they have virtually no credibility at all, which is a very sad state of affairs and will not be beneficial to the retail ForEx market in the long run. Thus the often heard statement by those such as myself that have learned this unpalatable truth first hand: if the EAs were REALLY profitable, then the authors would just be trading on them to make very good money as opposed to the perhaps even more difficult and very time, energy and resources consuming task of successfully mass marketing them on the internet and then (hopefully) all that is entailed in supporting them. While this is not universally true, it is to the degree that for all intents and purposes, it is the reality of almost the entirety of the retail commercial market.

While this forum is dedicated primarily to learning to program EAs, the underlying premise and foundation has to be not just making EAs that work, but that work profitably. This article is an example of this. We are not here just to learn how to make EAs that automatically lose lots of money. We can all do that readily enough by ourselves. Automating these only makes them worse. Let us have more focus on the ways, technique and valid algorithms that are profitable and THEN work on automating them effectively. One has to have a profitable system to start with, otherwise it is going to be an unprofitable loser regardless of if it is automated successfully or not. We need to have more such focus and content as this article in effectiveness and successful outcomes, not just proficient MQL programming.

This illustrates that unless one has a trading system, automated or otherwise, that makes a great many trades with a VERY high degree of accuracy, then it cannot have a substantially higher Stop Loss than the Take Profit. I believe that to have a robust, long term successful trading strategy, that the SL should be no higher and preferably at least a couple of times lower than the SL. Alas these seem to be few and far in between and are in the minority by a very large degree. In most case I have found that those that are the reverse, and are so because their is no statistical basis for there validity as a profitable EA without this, and even this does not usually make then profitable.. So they just make it trade more often, set the SL several times higher than the TP so that they make more winning than losing trades to utilize this statistic to market and sell, but which by itself is misleading; even though it IS true and dazzle one with 'the very high accuracy rate of all the winning trades' they are making for us. But that end being unprofitable 'drainers', 'bleeders' and 'losers' regardless.

Bottom line: if we don't know how to trade the ForEx profitably, then all the programming skills in the world aren't going to make money for us using these EAs to trade with ourselves. Their are many in this category that having learned MQL end up producing many unprofitable EAs and sell them to others which is how they make their profits with them: not on the ForEx itself. After all, isn't that why we are all here? Not just to become adept at programming in MQL, but to be effective and competent enough to take our FX winning strategies and then automate them. Do we want to spend years or decades of our lives becoming very proficient at MQL, only to be STILL losing money? Albeit more adeptly, readily and efficiently and at an even higher rate!

Good MQL programming skills are a means to an end, and not the end goal itself.

I want to know how to program PROFITABLE ForEx EAs, not just get proficient and efficient at programming in MQL.


Reason: