A great book on testing and optimisation - page 3
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Vita, you make a good point. I myself have big doubts about the sufficiency of the forward test to conclude about the sustainability of a strategy. It is only a necessary test, which essentially says: "If the strategy is sustainable, the forward test will be good". The equivalent statement is: "If the forward test is bad, then the strategy is bad".
The law of meanness: this is just a negative statement, which allows you to screen out unsustainable strategies - but does not allow you to guarantee the quality of sustainable ones. It's still not bad either, as 90-95% of blatantly bad strategies are screened out...
I have been thinking about it for a long time but haven't managed to write it as an article. To tell you the truth, I'm tired of writing it, because new problems keep popping up. But I can tell you that my own approach differs greatly from Pardo's.
And in general, Vita, sufficient conditions ("If some test group is good, then the strategy is good") is an unattainable goal, you understand. What positive guarantees can there be? They do not and will not exist... But you can improve as much as you like, adding more and more negative criteria (necessary) to ensure that bad strategies are cut off...
Nevertheless, I think this book is a very good introduction to the extreme complexity of the problem. The author doesn't actually solve it, he only shows the state of existing approaches to it, but it's much better than the solitary optimization, the results of which grail creators post here.
Heh, I thought the scientific method would have to give guarantees. Looks like the author is unscientific after all. See the same point 3. "Exceptionally unfavourable and untested market conditions have arisen" is one of the loopholes to justify failure while providing facts on forward testing that contradict the book. That's the way astrology, alchemy and other chiromancy works - piling up conditions, sometimes impossible to meet and no guarantees. You see, "the stars didn't align in the sky that way..."
With the article, Godspeed.
How do you feel about the following, as it seems to me, very simple rule for sifting out "unstable" strategies - never test with take and stop of different size?
Vita, you are right about astrology: all trading is astrology. But there are still authors who try to turn astrology into astronomy...
If you cannot find sufficient conditions for a very long time, you can try to find a set of necessary conditions that is as close to sufficient as possible. In principle this is one of the heuristics leading to a scientific approach. The market is an introspective monster that will always resist attempts to know it scientifically: any strategy that works at a certain point ceases to work later, once it becomes common, since it is still a roughly zero-sum game.
I haven't thought about the take and stop, but it's an interesting idea. This is another dimension of optimisation.
The link doesn't work, though :(
Moscow time 13.08.2007 20:45. Tried both downloader and Firefox, link is broken.
but in a more sophisticated (confusing and incomprehensible) form :))
Any optimization is a series of tests, and then selection of the best option in our opinion.
WFO fits completely into this definition.
The only difference is in the criterion to be used.
It turns out different for WFO.
And optimization on a fixed part of the history together with multimarket analysis on it is, in fact, also a complete mess... Frankly speaking, the book actually lists a set of tests, between which there is no internal logical connection, i.e. no single concept permeating the entire set of procedures.
Nevertheless, I will stress once again that the book is extremely useful for those who are used to baking grails in packs - simply because it gives an idea of the whole complex of the most difficult problems of strategy evaluation.
P.S. I can imagine such a forum thread: "I'll buy a testing/optimization technique that gives a 100% guarantee of strategy performance"...
P.S. Imagine a forum thread like this: "I'll buy a testing/optimisation methodology that gives a 100% guarantee of a working strategy"...
Heh. Wanted. Methodology will not sell, but the service "Testing / Optimization, giving a 100% guarantee of (non) performance strategy" give.
I take back what I said about the bullshit nature of the forward analysis described in Pardo's book. Provided that all the tests preceding it are passed successfully (maybe not counting the multi-market one, which is hardly needed if you play a single instrument), the forward analysis is quite reasonable, because it replicates the actions of a real trader in a real trade. The actual optimization part of WFA was implemented by xeon, but of course this is only one small stage of complete work, i.e. optimization on optimization data - without testing EA with optimized parameters on out-of-sample data. Of course, Vita, and here the chances of failure remain, as successful forward analysis is only a prerequisite for the quality of the EA.
I take back what I said about the bullshit nature of the forward analysis described in Pardo's book. Provided that all the tests preceding it are passed successfully (maybe not counting the multi-market one, which is hardly needed if you play a single instrument), the forward analysis is quite reasonable, because it replicates the actions of a real trader in a real trade. Actually optimization part of WFA was implemented by xeon, but of course it is only one small stage of complete work, i.e. optimization on optimization data - without testing EA with optimized parameters on out-of-sample data. Of course, Vita, and here there are chances of failure, because successful forward analysis is only necessary condition of EA quality.
If nothing else, I will soon complete the library which will allow me to implement "forward analysis" to the fullest extent.
Of course, Vita, and here the chances of failure remain, as a successful forward analysis is only a prerequisite for a quality EA.
In my opinion the only obvious way out is diversification (by model, by parameter, by instrument) and the result will be non-random. Law of large numbers :))))))))))
And in the process of trading you may find tactics, which clearly are losing money and re-optimize or even remove them from the system.
Xeon, Matemat, Vita, tell us how you optimize your systems!!!! How do your methods differ from Pardo's?
In my opinion this is a very important question, for me perhaps there is no more important question right now: ))))))
If nothing else, I will soon finish the library that will allow to implement "forward analysis" in full.
How will the library work please tell??
I have an idea of search of consistency via FT, here is the following:
I.e. we take three test windows (TO) on each of them we do optimization, save results of optimization on each TO, then we choose on one of TO the runs in descending order of profitability. We take the number of a run, find runs with the same number in two neighboring maintenance shops and if profitability of these two runs satisfies us
we assume that runs with this number have passed.
Apparently, there is a pattern of "you win in profitability, you lose in consistency", and vice versa. What do you think?