Optimisation range
I would like to ask a question about the data range used to optimise EAs. I.e. on which timeframes which ranges to choose. For example, for H1, is it enough to optimize the Expert Advisor on monthly, triennial, and annual data? I would like to see these values for different timeframes and at least a brief substantiation of the choice.
There are no clear criteria; everyone has an opinion. I will try to express mine.
Optimization over the entire period of history is pure self-deception. That is why the optimization interval must be at least half of the testing period. However the smaller portion of the testing period is used for optimization, the better. After selecting parameters at the optimization interval, run the test on the full interval that you selected as the testing period. You can start "believing" the obtained results, if the sample amounts to at least 100 trades. But even that is not enough for many people. It is clear that a larger amount of deals gives a more accurate picture of the system. For example, I can "believe" in a sample of at least 200 deals. 1000 trades is optimal, but there is no limit to the perfection...
This issue is very well dealt with by Jezev and Shumsky. Here is what can be said by summarising what they have done:
The task of learning theory is not to minimise the learning error, but to minimise the generalisation error defined for all possible future examples. This is how an optimised TS will have maximum predictive power. And the difficulty here is that it is and only the learning error that is actually observable. The error of generalisation can only be estimated, based on some considerations. The errorof the tester's predictions on the new data is determined by the total length of the model's description of the data, along with the description of the model itself .
Approximation error
Since the number of decisions made by the TC, as a rule, is much smaller than the number of parameters adjusted in it, in this case the average error of approximation will be expressed through the total number of fitting parameters w as follows Eapprox= d/w, where d is the number of input parameters that the TS analyses. In the vast majority of cases d=w is fulfilled (all the fitting parameters are concentrated at the TC input).
The main problem of learning algorithms, is known as the problem of over-learning or fitting to history.The essence of this problem is best explained by a concrete example. Suppose that training examples are generated by a function we want to reproduce. In the theory of learning, this function is called a teacher. Given a finite number of training examples, it is always possible to construct the test run with zero learning error, i.e. with the error defined on the set of training examples. To do this, one has to take a TS with the number of fitting parameters greater than the number of examples. Indeed, to reproduce each example we have P equations for W unknowns.And if the number of unknowns is less than the number of equations, such a system is underdetermined and admits infinitely many solutions. This is the main problem: we do not have enough information to choose the only correct solution - the teacher function. As a result, the randomly selected function yields poor predictions on new examples that were not present in the training sample, even though Tester reproduced the latter on history without error. Instead of generalizing the known examples, he memorized them. This effect is called overlearning.
Error related to the complexity of the model.
The description of a model basically comes down to passing the values of its sub-adjusted parameters. For a given accuracy such a description will require about W bits. Consequently, the specific error per example related to model complexity can be estimated as follows: Ecomplex=W/P. As we see, it monotonically decreases with increasing number of examples (length of history in training).
Indeed, in order to unambiguously determine fitting parameters by P given examples, it is necessary that the system of P of equations mustbe overdetermined, i.e. the number of parameters W must be greater than the number of equations. The larger the degree of overdetermination, the less the learning result depends on the particular choice of a subset of training examples. The generalization error component defined above is precisely related to solution variations due to the finiteness of the number of examples.
So, we estimated both components of strategy tester generalization error : E=Eapprox+ Ecomplex=d/W+W/P>SQRT(d/P).Importantly, these components depend differently on the number of fitting parameters W, which implies that the optimal size that minimizes the overall error can be chosen. Minimal error (equality sign) is reached at optimal length of historical data participating in strategy optimization: P=k*W^2/d=k*W, where k-constant is approximately equal to 4.
Thus, the minimum prediction error of an Expert Advisor in the forward test optimized in the strategy tester on historical data will be achieved if the historical data length is four times greater than the number of fitting parameters in the strategy tester.
If we optimize 5 parameters in the tester (e.g., Mach periods), then the optimal length of history should be such that the tester would use 4*5=20 transactions on it. It can take from 1 to ...200 days of history, it all depends on the adopted strategy. Reducing of this number will lead to fitting the tester to the history, and increasing - to deterioration of approximation quality and as consequence, to deterioration of the forecast accuracy.
There are many ways of knowing this World.
Some get to know it through Love. Some through meditation, etc. I gave a way of estimating the value of interest through logic and mathematics. So your secret is not a secret at all, it is just another facet...
P.S. What I talked about above is just one act of optimizing the parameters. To assess the performance of an optimized EA, one point is not enough, you need to collect statistics, while statistically significant result starts with a hundred runs on independent data with overoptimization at each pass ...
So, the recipe for adequate optimization in the tester won't be simple.
Thank you all very much, especially Neutron for the detailed description. Recently I found a very interesting idea on the forum and I am working on implementing it in my code. At the same time I tested it on demo and got quite good result, about 150% after 3 days, but there are some subtleties... In order not to create a new topic I will ask here:
When defining an entry point it is necessary to find out if an order is already open in the same direction at a distance of n pips. The following code does not work but I cannot understand why:
bool buy=true;
for(int cnt=0;cnt<OrdersTotal();cnt++)
{
OrderSelect(cnt, SELECT_BY_POS, MODE_TRADES);
if(OrderType()==OP_BUY && OrderOpenPrice()<=Bid && Bid-OrderOpenPrice()<50*Point) buy=false;
}
if (buy) OrderSend(Symbol(),OP_BUY,0.1,Ask,3,0,0, "MyAdvisor",0,0,Blue);
As a result, when I run an EA with 10 possible open lots, it opens them all from one point. I want it to open one order and only open the next one if the difference between the current price and the price of the current open order is more than 50 pips
to Neutron
do these two types of errors apply to any TC or just neural networks?
budimir You choose for the definition of parameters the trend areas? Do you choose the trendlines and the side trendlines?
I select the most lively trend sections, and the fitting section turns out only 20-30 deals, while the sample on the fitting section should be at least 100-1000 deals for
maternal reliability - all this is nonsense, well, let's say you took a fitting plot, directly adjacent to the future, identified the phase of the market - a trend or it is a sideways trend or a change in direction, and then? nothing! The market is NOT stationary, and we don't know where it will go in the future!
to ITeXPert > don't make fun of the order of 150% in 3 days.
and in my TS you can get more than a dozen of such lots in 3 days with 150% ! :о)
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
Hi all!
I would like to ask a question about the data range used to optimise EAs. I.e. on which timeframes which ranges to choose. For example, for H1, is it enough to optimise an EA on one month, three months, or one year of data? I would like to see these values for different timeframes and at least a brief substantiation of the choice. Thank you very much.