Trader's self-deception: distrust of forwards. - page 3

New comment
 
khorosh:
Forward is a useful check, but not a panacea. In the optimisation results it is possible to select a variant of the parameters providing a good forward, but this does not guarantee good results on the real.
Suggest a better method of analysis.
 
Youri Tarshecki:
Focusing on the LAST changes is another kind of self-deception. Of course, you have to use fresh settings to trade. But if you limit yourself to optimizing for the last period, you will never be able to understand whether your Expert Advisor is resistant to future changes. Look at the second picture in my post - you can clearly see it. Forward is a comparison of the past with the past. Your method has nothing to do with forward analysis. Virtually any Expert Advisor can be optimized to a nice backtest. The pattern here is as follows - the more settings are optimized, the more beautiful the backtest is. And the uglier is the forward. Forwards help to find exactly the logic that uses more long-term patterns, and therefore is more promising.

I don't see a reasonable doubt - your assertions are without logical conclusions.

Prove that optimization over last 3 years and check it on historical data (3 years before the optimization, for example) is a worse method than optimization on 3 years of 6-year history and the subsequent verification of the optimization results on the last 3 years?

In any case, my suggestion to modify the EA, allows to optimize on both 6 year old data and 3 year old data, at once.

 
-Aleks-:

I don't see a reasonable doubt - your assertions are without logical conclusions.

Prove that optimization over last 3 years and check it on historical data (3 years before the optimization, for example) is a worse method than optimization on 3 years of 6-year history and the subsequent verification of the optimization results on the last 3 years?

In any case, my suggestion to modify the Expert Advisor, allows you to optimize both on 6 years old data and on 3 years old data, at the same time.

It's not about the direction of checking. It's about the number of forwards. If you prefer to look for "reverse inertia" instead of normal inertia, in your case you can only do it once. - You're pinned down by the last stretch to the present. Doing a single, albeit "reversed" forward is too unreliable. In addition, the "check" segment is twice as far away from the actual purpose of this check, the real trade, because the back segment is not a check. But that's not the main point. What you are doing is a particular case of cross forward comparison. I have not done it myself, but according to some reviews it has the same effect as successive forwards, and it works only when there are statistically many of them. And you only have one. And in terms of MT it is not clear at all how to automate such a method.
 
Youri Tarshecki:
It's not about the direction of the inspection. It's about the number of forwards. If you prefer to look for "reverse inertia" instead of normal inertia, in your case you can only do it once. - you are pinned down by the last stretch to the present. Doing a single, albeit "reversed" forward is too unreliable. In addition, the "check" segment is twice as far away from the actual purpose of this check, the real trade, because the back segment is not a check. But that's not the main point. What you are doing is a particular case of cross forward comparison. I have not done it myself, but according to some reviews it has the same effect as successive forwards, and it works only when there are statistically many of them. And you only have one. And in terms of MT it is not clear at all how to automate this method.
Again, I do not understand the beauty of splitting a year into months and averaging the result? It is obvious that we must consider the market condition - trend or flat - during analysis, and the more such conditions, including changes of one condition to another, the more understandable and reliable the picture will be. Or is your Expert Advisor working on minutes?
 
-Aleks-:
Again, I do not understand what is the beauty of splitting a year into months and averaging the result? It is obvious that one should take into account the market condition - trend or flat - during analysis, and the more such conditions, including changes of one condition to another, the more clear and reliable the picture will be. Or is your Expert Advisor working on the minutes?
Once again I ask you - why aren't you optimizing on ALL of the available history? According to your logic, then the picture will be the most accurate. The correlation of timeframes for analysis is an experimental one and it does not matter at all whether the EA works on minutes or not. The experiment itself reveals in which mode there is less drawdown and more profit. Moreover, recently I've been having some thoughts that each variable must have its own optimal optimization interval, but it is probably the thing of the future.
 
Youri Tarshecki:
Once again I ask - why don't you optimize on ALL of the available history? By your logic, the picture would then be the most reliable. The correlation of timeframes for the analysis is determined experimentally, and it does not matter at all whether the Expert Advisor works on minutes or not. The experiment itself reveals in which mode there is less drawdown and more profit.

I optimise some EAs throughout history (since 2000 usually) and I don't see anything wrong with that.

I understand that the market is changing. There are more stable behaviour patterns and there are less stable ones, but the closer this pattern is found to the current history, the longer it will serve - so I don't see the point in optimizing on old data without considering the current data. I only check satisfactory results on past data to check the stability of the logic, I don't expect the data to be better on history, but I do see a trend that allows me to assess the stability of the pattern.

Nah, I honestly don't see the point of breaking the history down into small sections without taking the graph into account. But if we group the history by trend and flat and test the Expert Advisor on them, for example if it works with the trend but has poor understanding of switch to flat, then it makes sense.

Again I don't understand why the suggested method of automation is not suitable for you?

I will say it again - we have to mark certain times in our EA and set a time frame in which we will not open orders, for example

Если (Дата_Открывать==1) Дата_старт=01.01.2015, Дата_стоп=10.05.2015;

Если (Дата_Открывать==2) Дата_старт=01.01.2014, Дата_стоп=31.12.2014;

Here we have an automated forward.

Then we cut the stripes in Excel - 5 minutes, plug the prepared formulas, and we are happy.

Maybe I do not understand correctly the optimization problem?

 
-Aleks-:

There are more persistent patterns of behaviour and there are less persistent ones, but the closer that pattern is found to the current story, the longer it will last -

That's true. But provided that you actually found that pattern and not a trivial fit has occurred. Only repeatability can give confidence, and forward in a single instance will show little, if only because the forward is usually worse than the backtest. I.e. comparing backtest with forward, and even in a single instance, is generally meaningless. I'm not, by the way, insisting that the segments should be "crushed" or "enlarged". I was stressing that only practice will give clarity, but the more frets you take, the more clarity will come. And about the cutoff itself - I don't know which answer would be better in your case, maybe the cutoff would be even bigger than you think.
 
Yuri_Evseenkov:
Suggest a better method of analysis.
I haven't thought of one yet).
 
Stanislav Korotky:

As far as I remember, this is what I did - I saved the time of the first trade into the frame - this is the "balance". Accordingly, when recording parameters and indicators of all the runs in one file, it is easy to distinguish the backtest from the forward by two distinctive dates. I have saved all runs in the same file (opened in OnTesterInit and closed in OnTesterDeinit).


It's already more interesting. But my situation is more complicated. Suppose I try to distinguish back and forward by date. But I'll have 20 backs by number of variables during one interval of history, and one forward. Since I've got 12 segments, I'll get plenty of dates for one year (240 backs +12 forwards) and I'll just try to recognize them there. Another solution is needed. For example - we need to add in the code the ability to save report not totally for each run, but when some external EA's signal appears. Suppose the automated optimizer that initiates all of the runs may write some conditional figures into a linked file. And the Expert Advisor reads them and saves the data strictly in those places where it is required. For example, B-8-2 (back table, 8th variable-row, 2nd run-column), F-3-11 (forward table, 3rd criterion-row, 11th run-column).
 

Everything is done in a much simpler way:

For example:

There is an EA. It has 10 settings - 10 sets. Optimization is the essence of choosing one of the ten sets. Right?

We run all variants on the entire history and lay out the results on the same even time scale.

Voila. Now we can poke at any point in time, analyse to the left of it to the desired depth and choose the set we like. This will be "optimisation".

Next we look to the right of this point - this will be "forward".

12345678910...17
New comment
Reason: