TSR - resuscitating trading systems
I disagreed with Reshetov in many ways (not on this topic). He and I occasionally docked an exchange of views, and usually he would send me away, calling me an idiot and a moron, either explicitly or implicitly. For my part I did not take it personally.
I have a big request to everyone: do not engage in fluderasty in this thread. If you have something constructive to say on the subject, write, and if not, don't write.
Constructive? Who's looking at the OOS from behind? That's from the obvious.
Put OOS at the front - no one is stopping you. The point is that in TS the reason comes first - a trading signal, and then the consequence - an open order that will be closed with profit or loss. And therefore, no matter whether it is in front of the OOS or behind, on the cause-and-effect relations, it will not affect anything, because the signals are not set up on OOS, but only checked there for lousiness. Trade signals will still precede market entries, and not vice versa, i.e. the order will not change. If the Expert Advisor is losing, it will lose both to the front and to the back. If it is profitable, it won't care where exactly the OOS is. The most important thing is that the OOS area is adjacent to the fitting areas. And since markets are constantly changing and the profitable plot has a limited shelf life, it is understandable that the OOS is better checked at the back, since the future on which we are going to trade is ahead.
So, to be more confident, we can test OOS both in front and behind. If the TS is losing here or there, then it is certainly not robust.
Yuri, in a few sentences, what is the essence and conclusion of your findings?
Yuri, can you briefly explain, in a few sentences, what is the essence and conclusion of your findings?
1. Even from the fitting, false signals can be identified and eliminated to a greater or lesser extent. But to do this we will need at least two different plots on which the optimisation was carried out. From them, by comparing for (un)coincidence of readings, we cut out false signals from potentially profitable ones. It is clear that it is impossible to cut off all false signals without exceptions, i.e. some part of them will be anyway lost. But it is quite enough for some TS to obtain an advantage in the form of a positive mathematical expectation.
2. Many TS were previously rejected, just because they fit well, but in forward testing gave unsatisfactory results. Now, having analyzed these TS using the method described above, it turns out that some of them are quite good working horses. Thus, some part of trading systems can be reanimated this way.
3. If you want to understand the essence of the methodology, you'll see that it is more reliable than a simple optimized and tested OOS.
... it's understandable that it's better to check at the back on OOS, as the future we're going to trade on is ahead of us.
As for me a couple of years ago I checked if OOS "behind" gives something "in front". And came to the conclusion that the correlation is not at all obvious. There are no particular guarantees. The probability of success of the TS may slightly increase, but the Expert Advisor can fail to give profit and lose it. We need other methods here.
Once again, using the above method no one forbids to check on OOS both front and back. If there is no profit on any of the forwards, the TS is not robust and it is better to get rid of it instead of trying to reanimate the stillborn. About the difference of results on different OOS, no one predicted 100% match.
As for guarantees, I have not promised them to anyone. Everyone has to test my suggested TS on OOS and demo or cent accounts and decide for himself whether he is satisfied with the result or not.
I would not be surprised if you spend your whole life in a vain search for a method with guaranteed results and die without finding it.
Continuing the theme: Where is the line between fitting and real patterns?
TSR is an abbreviation for Trading Systems Recovery ..................
As we have seen, although both perceptrons did not pass the forward tests outside the optimised sample of historical data, nevertheless the filter of their joint signals, gave positive results on history.
Yuri, I'm delighted. Even a slight envy (wandered a step away from this idea a couple of years ago) doesn't dampen the pleasure. It made me happy. I will experiment further.
... no one forbids checking for OOS both front and rear. ... If ... the TC is not robust and it's better to get rid of it ... there will be a difference in the results of the different OOS, so there's no promise of a 100% match.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Continuing the theme: Where is the line between fitting and real patterns?
TSR is an abbreviation for Trading Systems Recovery
I'm posting without a description of the theory and without screenshots. Screenshots will be in Code Base
And I will warn those who are especially gifted:
1. The attached version of the Expert Advisor has been cut to a minimum suitable for educational purposes
2. On the issues of improvement of the Expert Advisor, go to the BFG.
If you have any questions, like: "Why do I get different results?" No answer from the author.
Let's make an experiment using a trading system which is based on forecast of the future direction of quotes movement by adjusting of weighting coefficients of the elementary one-layer neural network - perceptron on historical data. The principle of this trading system was described in details in my article "How to find a trading system". Let's take historical data for the EURUSD currency pair for 9 previous months or more on H1 timeframe chart. We will divide it into three independent sections of three months each. The first one will be used for the final test, while the other two will be used to fit the history. To avoid running the trading system separately, I immediately combined two perceptrons in one trading system. And I created the function Supervisor() with the help of which the trading system has three modes of operation, depending on the input parameter pass: 1 - fitting and testing of the first perceptron, 2 - fitting and testing of the second perceptron, 3 - filtering by sifting out inconsistent readings of both perceptrons in testing mode without optimization or in auto-trading mode on a demo or real deposit.
Adjustable to the historical data are perceptron weights x11, x12 ... x42 as well as p and sl. The input parameter sl is a constant for all sections. Stop Loss and Take Profit levels are set for all open positions by this very value. Another input parameter p - the lag time period for the opening price difference, is also a constant. Entry into the market is made at the beginning of a new bar formation, i.e. according to the bar opening prices and perceptron readings, depending on the value of the pass input parameter, and exit only upon triggering of a Stop Loss or Take Profit. Optimization is carried out using the genetic algorithm of identifying extrema and maximum balance is taken as an extremum. The initial amount, for example $1000000, should be very large during optimization to prevent the algorithm from getting stuck on margin calls. Input parameters: lots - volume of positions to be opened in lots and mn - unique magic number, so that the Expert Advisor would not confuse the management of its own orders (which it has opened) with orders of others (which it has not opened).
At the first stage we need to find out what the values of input variables p and sl should be. To do this, we select the last two parts of the history, i.e. from 6 months ago till today. We set all parameters of perceptron weights to values from Start = 0 to Stop = 200 in steps of 1. Set the p-value from Start = 3 to Stop =100 in steps of 1, set the sl-value from Start = 100 to Stop =1000 in steps of 10 (or from 10 to 100 in steps of 1 for four-digit quotes). Set the pass value to 1. Tick the following parameters to be optimized: x11, x21, x31, x41, p and sl. All other checkboxes should be disabled. Activate the optimization. Once the optimization is completed, set the input parameters by the best pass.
The second step is to adjust the weights of the first perceptron on the second section of historical data. Set the optimization date and time from 6 months ago to 3 months ago. Uncheck the optimized parameters only from the input variables p and sl. Run the optimization. After the fitting is completed, set the input parameters according to the best pass.
Third stage: Fitting the weights of the second perceptron on the third section of historical data. Set the date and time of optimization from 3 months ago to the present day. Uncheck the optimized parameters: x11, x21, x31, x41 and set them for x12, x22, x32 and x42. The other checkboxes must be unchecked. Set input variable pass to 2. Start Optimization. Once the optimization is completed, set the input parameters by the best pass.
That's all, our trading system has been adjusted to the historical data from 6 months ago up to the present day. Let's save the values of the input parameters in the settings file. Set the pass input variable to 3. Uncheck "Use Date." Launch the test. We look at the testing chart. We can see that the balance and equity curve tends upwards in the right part of the chart and tends downwards in the left part. Now we must make sure that the balance tends upwards at the area outside the adjustment sample. We bring the cursor of our mouse to the balance line, where the rise of profit begins and look at the date in the tooltip. It turns out that the balance curve tended upward almost nine months ago, counting from today, excluding 10 days, i.e. 8 months and 20 days. And the adjustment was carried out on a stretch of 6 months. Hence, there is a successful test outside the optimised sample. We highlight this out-of-sample area to analyze it in more detail. On the whole the results are quite satisfactory, although considerably inferior to the record of J. Soros, but superior to that of W. Niederhoffer.
In order to make sure that we have dealt with fitting in some parts of the history, it is necessary and sufficient to uncheck the "Use date" checkbox. And run the test of the Expert Advisor with values 1 and 2 through the whole available history. In each of these modes, we can see that the upward growth of the balance curve is observed only within those periods, on which certain perceptrons were fitted. For all other periods of history there is no positive trend, except for individual humps ending in troughs.
As we have seen, despite the fact that both perceptrons did not pass the forward tests outside the optimized sample of historical data, nevertheless the filter of their joint signals, gave positive results on historical data of which nothing was known at the time of fitting. You can also experiment with other trading systems, such as those based on the breakdown of simple moving averages or on more advanced multilayer neural networks. If the trading system is robust, it is more likely to produce positive results on filtered trading signals outside the optimization period. If it is not robust, it will not give positive results on the optimized period with the filter enabled. However, the robustness of the TS is secondary compared to overhead costs of spread, swap and broker's commission. Therefore, with significant overhead, positive results in forward testing are just a dream, because the expected payoff is necessarily negative.