Why do algorithms stop working?
2019 also was a loss.
The world changes. So participants change their behavior in order to adapt to this changing world. That is why.
Nothing can be done about it. It remains a challenge to detect when a certain pattern no longer works, especially in automated trading.
The answer is very simple, but understanding the answer can be difficult.
The answer: Market conditions change!
The understanding: In your case, your strategy was using a fast/slow moving average cross, of a set period of your choice (e.g. 12 vs 26). But at some point, the market changed its characteristics and the periods you were using were no longer viable. If one were to consider the market as some fundamental wave, then you could say that its "frequency" changed, or maybe its "amplitude", or even its "phase", and the "filter" you were using was no longer able to filter out the noise from the signal.
I get that market change, can become more choppy in certain areas etc, but going from a huge gain to a significant loss from one period to another seems like such a drastic difference. Especially when the market wasn't choppy at all.
So even if one were to accept that 'things change', when then do you decide that an algo isn't profitable anymore and you stop using it?
More importantly, we make sense of charts by using indicators and if neccessary by using custom formulas to make sense of what's happening and to filter out the situations in which it's most favorable to trade. So that means that also for changing market situations, with the right indicators or formula, it should be possible to have an algo that adapts to these changing circumstances, as long as these changes in the market can be measured.
Do you guys have any indicators/formulas or other tools to measure these changes, other than the results of your algo itself? There's a solution for every problem and I'm hellbend on solving this before the end of summer.
I am not talking about choppy or trending markets. This is ordinary market behavior. I am talking about fundamental changes like changes in interest rates by central banks, starting or stopping QE programs etc. Not sure if you noticed but stocks and other assets took a big hit from the interest rate increase by the fed especially the so called growth stocks. You do not need a formula for that, you have to pay attention to the news.
I do not use any indicators. My algorythms automatically stop themselves when the results deviate from the history by a margin. There is no way to stop it before this deviation already has happened.
I am not talking about choppy or trending markets. This is ordinary market behavior. I am talking about fundamental changes like changes in interest rates by central banks, starting or stopping QE programs etc. Not sure if you noticed but stocks and other assets took a big hit from the interest rate increase by the fed especially the so called growth stocks. You do not need a formula for that, you have to pay attention to the news.
I do not use any indicators. My algorythms automatically stop themselves when the results deviate from the history by a margin. There is no way to stop it before this deviation already has happened.
When fundamentals change, it moves the markets either in the same or the opposite direction, or the market becomes choppy due to uncertainty. All of these should be able to get measured somehow.
The strategy I have works both ways. It buys when a certain short term MA crosses a certain longer term MA and then it sells when the opposite is true. So as long as there is momentum, MA strategies should still work to an extent. And the tests I've performed are for the Nasdaq, which had great momentum in the last two years especially.
I get that market change, can become more choppy in certain areas etc, but going from a huge gain to a significant loss from one period to another seems like such a drastic difference. Especially when the market wasn't choppy at all.
So even if one were to accept that 'things change', when then do you decide that an algo isn't profitable anymore and you stop using it?
More importantly, we make sense of charts by using indicators and if neccessary by using custom formulas to make sense of what's happening and to filter out the situations in which it's most favorable to trade. So that means that also for changing market situations, with the right indicators or formula, it should be possible to have an algo that adapts to these changing circumstances, as long as these changes in the market can be measured.
Do you guys have any indicators/formulas or other tools to measure these changes, other than the results of your algo itself? There's a solution for every problem and I'm hellbend on solving this before the end of summer.
It's the same reason why the algo didn't work during the 2019 bull market and suddenly work when the QE started. Programing engineers are supposed to be problem solvers, you left the 2019 problem unresolved and then it will come back again one day. It's a trade off that you chose to optimize your algo in the favour of the QE other than the 2019 bull market which at that time generated much better profit results in the tester.
I saw this kind of turning points in my scripts. I use part of the history data to build the algo, and when I apply the algo to the larger dataset for verification. I sometimes see a uptrend of the profit curve during the date range which I use the history data to build the script. Beyond that date range, everything becomes much worse. Then I know I added too many filters even with some fancy filters just because I got a better profit number. And sometimes when I got a really good result and started to verify it on the demo server, the date I built it or the day I ran it on the demo server was the turning point for the same reason.
By the way, in MT4/5's strategy tester, you could visualize the trades which you can compare the trades before and after the turning point. It could give you valuable information.
For a while now I've been working on creating an algorithm based on a short term moving average crossing a longer term moving average (and a bunch of filters on top of this).
Initially the result can look great. However, when I then went back in time I always see that the algorithm that worked so great before, all the sudden stops working or even generates a significant loss.
<Deleted>
Can someone explain to me why these things happen? I understand that momentum is a requirement for a good profit with moving average crossovers and also the ATR is important, but both were still fine for this symbol in the last year.
Any help would be greatly appreciated as I'm trying to solve this issue.
Hi,
In my opinion, there is one main reason for this. Overoptimization.
You even wrote the problem yourself. "(and a bunch of filters on top of this)".
Yes markets change, but if you have a system that uses the basic market characteristics (reversal, trend, momentum) then the strategy will still work in 20 years. The more you optimize, the more susceptible a strategy is to market changes. So when people write that a strategy doesn't work anymore because of market changes, it's because that strategy has been heavily optimized. That doesn't have to be a bad thing, but then you have to re-optimize the strategy more often.
Traders are constantly trying to create a nice backtest, as soon as the strategy is then traded live, this then behaves differently or in most cases even suddenly makes a loss. In this case, it has nothing to do with the change of the markets. The strategy was over-optimized and over-fitted to historical data.
When I develop strategies, I optimize a maximum of 4 parameters. Better are 3.
You write that your strategy consists of a moving average crossover entry signal. You have already used 2 parameters probably for the period. If you now add "(and a bunch of filters on top of this)", this is already critical. After that, you might still have an active SL and a TP. Furthermore an extra exit signal?
With so many parameters, I can create the most beautiful backtest in the world for you. But outside of the tested data, the strategy will almost certainly lose money. And this is not because the markets have changed overnight ;D
This is just my opinion on the subject....
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
For a while now I've been working on creating an algorithm based on a short term moving average crossing a longer term moving average (and a bunch of filters on top of this).
Initially the result can look great. However, when I then went back in time I always see that the algorithm that worked so great before, all the sudden stops working or even generates a significant loss.
<Deleted>
Can someone explain to me why these things happen? I understand that momentum is a requirement for a good profit with moving average crossovers and also the ATR is important, but both were still fine for this symbol in the last year.
Any help would be greatly appreciated as I'm trying to solve this issue.