A great book on testing and optimisation - page 8
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
we need to have a chat about Bernouli...
papaklass, here is the link, thanks to Yury Reshetov: http://bigfx.ru/load/8-1-0-4 . Doesn't seem to be broken, I just checked it.
http://nfkgtu.kai.ru/download/plugin.rar - download and put it on.
The link is broken.
I can share the unbroken one: http://bigfx.ru/load/8-1-0-4
Thanks for the link...still no email from the forum about registering...
It seems to me that the forward analysis (out-of-sample test) shows only that you have managed to fit the system so that it shows positive results in the out-of-sample test during the optimization process. Based on this statement we can draw the only correct conclusion - the probability of the system to make a profit in the future is 50% - it will either give or not give. :)
If you want to check it, you should do the out-of-sample test, and if you are satisfied with the results and you think you've found a working system, then do one more "out-of-sample" test on a doubled period. If you are satisfied with this test, do one last test. If you are satisfied with this test, you can say that you have found a working system. But will it work further ?!??, to check it you need what? .... right - one more test :)))))).
Isn't it so?
Another thing that comes to mind is about fitting systems:
Let's say I have a system and I fitted it to a curve so that it would give good results both in the test period and in the out-of-sample test. I adjusted it very simply - in the optimizer, setting many parameters, with very small step to get the best fit. Out of all possible fittings, in the same optimizer, using forward test I've got the best one. I'm glad, because now I have a system, though it's a good fit and unlikely to work.
My neighbour, Vasya, is a great mathematician and his approach is different. He takes exactly the same system, tests, makes a bunch of calculations, a bunch of tests, all scientific, and eventually gets the same system that I did, only using different methods.
So now the question arises - who has the system and who does not?
And the other question - how will the community react when they find out how I got my system? ... That's right - it's a pure fit and it won't work. And how will the community react to Vasya's system and how he got it? Right - well done Vasya, you're so smart and now you have a working system, because it didn't fit the curve, it passed all the tests and fit all the frameworks and parameters.
And the last question - who has a system that will really work? Me or Vasya?
It seems to me that the forward analysis (out-of-sample test) shows only that you have managed to fit the system so that it shows positive results in the out-of-sample test during the optimization process. Based on this statement we can draw the only correct conclusion - the probability of the system to make a profit in the future is 50% - you will either make a profit or you will not. :)
If you want to check it, you should do the out-of-sample test, and if you are satisfied with the results and you think you've found a working system, then do one more "out-of-sample" test on a doubled period. If you are satisfied with this test, then do one last test. If you are satisfied with this test, you can say that you have found a working system. But will it work further ?!??, to check it you need what? .... right - one more test :)))))).
Isn't it so?
I agree. I have come across that there is an "out-of-sample" fit too. And this fit is harder to understand and avoid......))))
Hmm, for some reason I was expecting criticism of a different polarity :). I thought my conclusions would be blown to smithereens.
LeoV is right that this kind of fitting is harder to understand. Many people don't even try to understand it.
Hmm, for some reason I was expecting criticism of a different polarity :). I thought my conclusions would be blown to smithereens.
LeoV is right that this kind of fitting is harder to understand. Many people don't even try to understand.
Those who will try to trash these conclusions, they do not understand anything yet. Some get it quickly, others for 10 years still do not understand. I.e. it is all the same plummers who are engaged in masturbation under the name - optimization, and train neural networks for it is not clear what. They look for where there is more light, and not where it is necessary.
I agree. I have come across that there is also an 'out-of-sample' fit. And this fit is harder to understand and avoid......))))
And why so uncertain? Out-of-sample,forward and other crap is clinical.On martingale there is no way to avoid the fit.