Principles of working with an optimiser and basic ways of avoiding fitting in.
I would love to support this thread by compiling my scattered statements on the forum.
C-4:
Testing on historical data is necessary, but not sufficient, to ensure the likelihood of a positive outcome of your trade in the future
I completely disagree with this statement.
As such, testing has nothing to do with predicting the probability of future trade outcomes. It is simply a figure that assesses the performance of the TS in this particular area and that's all.
The tester is an excellent means of debugging the TS as a program, but it does not give us anything for estimating the essence of the trading algorithm.
A common example.
A trader naturally walks and covers 1 km in 10 minutes. Is it a guarantee that the trader will walk the next km in 10 minutes? We are told to do a forward test and walk 1 km more. We did it in 9 minutes. So what? What's the implication? In my opinion, exactly nothing, because the tests only show that the trader will run one more kilometer he has covered. If future km are similar or have similar characteristics to the passed one, everything will be OK. But what if the next km is a river or a bog? Obviously the trader's movement on this new terrain will be different and the results of his tests have nothing to do with predicting his movement on the water.
This simplest analogy, which shows the location of the test as such, is nothing more than a timing device when walking.
Now the optimisation and retooling.
This is the trainer who trains our trader walker. At the stadium )it's such an area) this trainer taught our walker to walk in 9 min, then in 8 min etc. Where he had to stop his optimisation so as not to overfit our walker. That sounds ridiculous. Probably hinting at the use of doping.
But our coach is unaware that the walker will have to walk on a different terrain. Our coach, and with him the walker, are not even aware that there are other terrain options. They are both plotting and shamanising, considering TA as science and the tester and optimiser as proof of the results of their kamlana about the future.
From the two posts, the conclusion follows:
on the one hand we have to describe the terrain options
on the other hand to accurately describe our walker
having these two descriptions be able to compare these two components. Do we need a tester here? Yes, we do. Is it the main one? No, it is not, because it's just an arithmometer and the problem of TC stability lies entirely on a different plane.
C-4:
I completely disagree with this statement.
Everything you write is correct, as long as we are working with just one of the market-fitting functions. We are essentially writing about the same thing. What does the optimizer have to show? That the function is being adjusted? It will show this in the form of a systematic increase in profit.
It is quite conceivable that the result of the strategy can be computed analytically. For example, we can go through the inflection points in the optimiser and find the best ones, or we can solve this problem through a system of linear equations and place these points in the same places. The optimiser as well as the analytical solution is only a search method. But the result of the search will be the same.
But I don't agree that the optimizer doesn't give you anything new. Every tool has its own advantages and the optimizer is no exception. Its main advantage (apart from simplicity and usability) is the shift method. You can describe your model and apply it analytically to a certain piece of market history. But the optimizer can shift the parameters of this model from the point of your specified extremum by a certain distance. If the system's performance in this case drops dramatically and the shift distance is not too large, it is an obvious sign of fitting or finding a random non-existent dependence (statistical spike). Also, a shift can be made on the test data itself, e.g. forward in time, outside the training sample (OOS), or on a different instrument altogether (why should a stable pattern work on only one instrument?). The extent to which the performance of the system will change when using these methods will be the criterion of how promising the system can be. That is, the tester allows you to calculate the degree to which the system is stationary relative to its own parameters and time window of operation!
So why is the shear method so effective? And it is effective, because as soon as you change one parameter a bit in some strategies, or use other parameters, and all profits disappear at once (probably everyone has faced this). The answer is simple: this method is the closest to reality. Whether we like it or not: tomorrow's market will be a little different than today's. The process we are exploiting will also change slightly. That is, the shift will happen in relation to our fixed system, no matter how good the parameters are, and no matter whether we want them to or not. So, since we cannot avoid it, why not prepare for it in advance. The future changes are unknown to us, but we can "fix" the market condition in the past, and on the contrary, we can shift the values of the system parameters by the search method. If the parameters are stable, the result should not change much from the changes of these parameters, if unstable, the future inevitable shift will break our system.
Up until 10 years ago, traders checked their TS manually: they developed rules, applied them to quotes and calculated several results, as long as they had enough patience and perseverance. The conclusion about the TS suitability was made on the basis of this "testing". If there are too many losing trades (3-5), then it's discarded. Today, we will laugh at this approach, as we judge on the level of number of identical deals in a row and deduce the final figure based on several hundreds of trades.
You suggest moving the window forward and getting a lot of totals. As far as I know it cannot be done automatically in MQ4 tester and we repeat the experience of 10 years ago at the next level. But that's not the point, as nothing has changed qualitatively. How many of these totals do we need to get from the tester? Again, how much patience and assiduity is enough? And are these human qualities a guarantee of obtaining similar results in the future? The answer is unequivocally no. There must be some other judgement about the results, and that judgement is obvious: if the window shift results (of at least 30, and preferably several hundred) are stationary, then we can trust that we will get similar results in the future. If these tester results after a window shiftare not stationary - then any conclusions about TC are not appropriate at all! On a non-stationary series, we can only state the past - this is how it was, and how it will be - we do not know.
To quote the topicstarter (I'm not a mathematician and may be mistaken in pointing out the inaccuracy, please forgive me in advance for the lack of clever words and other water)
Основная проблема трейдеров жалующихся на нестационарность или постоянную смену рыночного настроения в том, что они ставят своей целью аппроксимировать состояние рынка, а не поиск закономерностей, которые на нем присутствуют.
Searching for patterns in non-stationarity is impossible, it contradicts trivial logic.
On the non-stationary series we can only state the past - this is how it was, and how it will be - we do not know.
Totally agree.
I have reread the author again:
The purpose of this thread is to share experiences between highly skilled traders.
To quote the topicstarter (I'm not a mathematician and may be mistaken in pointing out the inaccuracy, please forgive me in advance for the lack of clever words and other water)
Finding patterns in non-stationarity is impossible, it defies trivial logic.
It does not allow the application of statistics, and it does not contradict logic in any way. It is not necessary to look for patterns by statistical methods alone.
Contradictions to logic only occur when some botanists try to measure non-stationary data using statistical methods.
This does not allow for the application of statistics, but does not contradict logic in any way. It is not necessary to look for patterns by statistical methods alone.
Contradictions to logic only occur when some botanists try to measure non-stationary data using statistical methods.
You can measure non-stationary data with anything you like, it will not invalidate its non-stationarity. With all due respect Reshetov (no kidding, I remember puzzling over your future-shifted averages, really appreciate your thoughts)-but you used sophism. That's it, I'm not flubbing-you're too young to be clever.
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
It's been a long time since I've created new topics, but being on this forum for a year now, I see that a frightening number of people in our community of mts'niki traders don't understand or know the tools they need to work with. On the other hand, about half a year ago the forum administration asked me to write an article about the technology of creating a robust EA. Unfortunately, I did not manage to write a thorough article on this topic due to my busy schedule. So I hope this article will be a light version of basic recipes from professionals - for beginners, or anyone who wants to understand the finer points of working with statistics, optimization, searching for the right decisions, and finally finding a good robot that would be profitable not only in history, but in the future as well.
This article is written by professionals, not by me, at least only me. I fervently hope that this subject will be discussed by people who have understood the basic principles of working with historical data using their own little experience.
At the same time, simply to continue one of old branches I do not want, - in fact the initial message, a vector of development of a theme is important. Also, knowing the incredible ability of this forum to self-satisfaction, I would like to immediately define the criteria for communication. Very please - do not express your opinion on everything, just because you have it. The purpose of this thread is an exchange of experience between highly qualified traders. I especially want to emphasize the word "traders". Not programmers, not econometricians, not mathematicians (of course this does not apply to Alexei:), but traders. Even if you just listen silently to valid ideas - it will already enrich your knowledge piggy bank, and the quintessence of knowledge will not be diluted with a succession of consecutive opinions of people who still have much to understand. I say this without excessive pathos, narcissism and provocation. There is much I do not know myself and I openly admit it. My aim is the same as that of a beginner who knows for the first time what optimization is. I want to learn something new and interesting from people who know the subject better than I do.
Why do we need testing on historical data and whether it is needed at all is a question that many traders, some of whom are even mts-niks, ask. I'll answer right away:
Тестирование на исторических данных обязательное, но не достаточное условие, обеспечивающее вероятность положительного исхода вашей торговли в будущем.
Absolutely, as for every statement there are 1,000 caveats to be found here. Just do not write them here, I already know what you want to say. I would add that there is a class of investors whose investment horizon is huge (like Warren Buffett). They are not interested in what will happen to the market in the first 10-15 years. They do not work with the market, they work with companies. Their actions will never become statistically significant. It's just a different way of working, that's all.
Someone will say that any TS is a fitting for a specific market. It's true. Working with TS we work with certain rules of entries and exits. The purpose of these rules is to synchronize the position in the account with the current market trend. But on the other hand, a deep understanding of the processes that occur in the market and their proper use is the way to profit. If we understand and learn to work with it, we will not be afraid of market volatility. This is a "scary" property of the market, and in simple terms it means that the outcome in the past does not determine the outcome in the future. If any market property changes, any TS that exploits this property will instantly stop working.
I hope this thread will tell us how to recognize in time that the market has changed its properties, how to distinguish a pattern from the coincidence, and how to find and use it correctly.
As an introductory note, I will move a few previous posts that provoked me to start this topic. So let's begin.
C-4 : You clearly overestimate people with a good scientific background. They tend to put the scientific method ahead of the idea. Often there is nothing in their methods but the method itself. For them, the market is not important, it is only the object of application; it is the method itself that is important. They spend years fighting windmills that in fact don't even exist. Just generate simple and sensible ideas and then the market will reciprocate. I can safely say that the depth of an idea is inversely proportional to its complexity - the market knows this and proves it to me every day.
Ask : Are you saying (I'm paraphrasing you): the market is simple and you can make money on it using rather simple and primitive methods?C-4:Yes, that's right.
I fully and unconditionally agree with C-4.
Concerning this, I can explain the following.
CU can be either complex or simple. What is the difference between them? The difference is this -
The trader applying both complex and simple TS, eventually comes to the same end - he has to determine by some methods or his intuition how his TS will work in the future on unknown data, since the trader trades not on past data, but on future data, I think that everyone must agree with this. And no one knows what will happen in the future, especially neither the most complex nor the simplest TS.
What is a complex or simple TS? What are the pros and cons?
A complex TS has:
1. A large number of variables that can and unambiguously lead to a fit with past data, which consequently leads to a plum in the future.
2. To avoid fitting, some variables need to be fixed - which variables need to be fixed, how fixed?
3. Due to the large number of variables - a large number of combinations of parameters from which it is difficult to choose the parameters that will work best in the future. 2.
Difficult for a trader to grasp the principles of TS operation, therefore it is difficult to foresee the operation in the future.
A simple TS has:
1. Small amount of variables which allows avoiding fitting.
2. Fixation of some variables which can be comprehended and fixed in the right value.
3. A small number of combinations of parameters which can be conceptualised as the market changes in the future.
4. The work of the TS, which can be comprehended and understood how it may work in the future and if it will not work in the future, why. )))
The point of my post is that the problem for both complex and simple TS is the same - how it will work in the future.
Each successive parameter is an additional degree of freedom. Whether we want it or not, TS approximates instrument chart in one way or another. Our purchases will yield profits only when the price is rising, and vice versa, when the price is falling. Now let's represent the approximation of the graph by a linear trend. It can be drawn with just two points (parameters). It will indicate the general direction, but at each point in time, the price will be quite different from its value. It is very problematic to fit a linear trend closely to the price (econometricians will cry). Now build a second- or third-order power approximation. Their formulas have more reference points (parameters). Now our approximation is non-linear, it curves and follows the price change. Each of its points is much closer to a similar point of the price itself (econometricians rejoice). In the first case, a system that trades on a linear trend can only take its mathematical expectation on the history. In the second case, the system will take much more. Take a look at the chart below. If we make a system that flips depending on the slope of the approximating function, its result will be the better, the more parameters define this function and the longer and more complex the formula itself is.
As you can see, the system where more parameters are used will work on history. But our ability to set inflection points does not mean anything as all these functions describe not a regularity, but the market condition, and tomorrow the market condition will change and the function to be fitted will no longer fit it. The main problem of traders complaining about non-stationarity or permanently changing market mood is that they set their aim to approximate the market condition, not to search for regularities in it.
Now imagine that you have instructed the optimizer to arrange the inflection points. For the strategy operating on a linear trend you gave two points (entry and exit), for the strategy on the trend of degree 2 you gave three points (entry, exit and one inflection point), for the strategy on the sixth-degree trend you gave 5 points. What do you want the optimizer to find? It will find the inflection points so accurately that the final criterion is the best one. Many people don't understand this and force them to enumerate a six-dimensional space of possible parameters in small steps. The poor optimizer will be combining these points for several days, but finally he'll find what I've defined by eye on this graph at the rate of three interpretations per minute. There really is no problem with the speed of enumeration - this is just a special case of not understanding what is actually being searched for, and will be suggested as an answer.