Discussing the article: "Brute force approach to patterns search (Part VI): Cyclic optimization" - page 3

 
fxsaber #:

But to specifically look for beauty by Sharpe, R^2 or criterion in this article seems questionable. Maybe I'm wrong.

Very right
 
mytarmailS #:
Very right

Very right that something seems questionable )))) even so, but all this is a rattle, while someone is very right or wrong I have already turned it all into a product. What's with the terminology, very wrong..... These curves are just a way of pushing the standard deviation closer to the mathematical expectation, people, come on.... Working with a limited sample will force you to do this because it is the only way to increase confidence in that sample.

 
Evgeniy Ilin #:

Very right that something is seen as dubious ))) even so, but it's all codswallop, while someone is very right or wrong I've already turned it all into a product.

The fact of turning something into a product does not mean that it does something useful.

I'll answer the rest later.

 
mytarmailS #:

I'll answer the rest later.

Just tinkering with the TC to fit a nice profit curve is overtraining, even if you have OOS

Are you familiar with the multiple testing error?


Familiarise yourself with these materials


P-hacking and retraining backtests " Mathematical Investor (mathinvestor.org)

How overtraining on history in finance leads to false discoveries " The Math Investor (mathinvestor.org)

Backtest history overtraining and the post-hoc probability error " The Mathematical Investor (mathinvestor.org)

backtest-prob.pdf (davidhbailey.com)

AI in Finance: how to finally start to believe your backtests [3/3] | by Alex Honchar | Towards Data Science

Demystifying the Probability of Backtest Overfitting: A Step-by-Step Guide with Python Code and Visual aids | by Francesco Landolfi | Python in Plain English


And then you will realise that you are just overtraining, no matter what criterion you use :

away to push the standard deviation closer to the mathematical expectation or the slope of the regression line or profit maximisation or Sharpe or.....


What to do:

You are now just fitting a nice curve through many iterations, the error of multiple testing shows that even at random you can build a TS that will show a nice curve on both test and traine.


And it is necessary to

1) Develop a system of simulations, confidence intervals and take the curve as a result of not one calculation of trading TS as you have, but for example 50 simulations of TS in different environments, the average of these 50 simulations to take as a result of the fitness function that should be maximised/minimised.


2) During the search for the best curve ( from point 1 ) by the optimisation algorithm, each iteration should be correlated for multiple testing.

The problem of multiple testing in practice / Habr (habr.com)


That's how it is...

 
mytarmailS #:

Just tweaking the TC to fit a nice profit curve is overtraining, even if you have OOS

Are you familiar with the multiple testing error?


Read these materials


P-hacking and retraining backtests " Mathinvestor (mathinvestor.org)

How overtraining on history in finance leads to false discoveries " The Math Investor (mathinvestor.org)

History test overtraining and the post-hoc probability error " The Mathematical Investor (mathinvestor.org)

backtest-prob.pdf (davidhbailey.com)

AI in Finance: how to finally start to believe your backtests [3/3] | by Alex Honchar | Towards Data Science

Demystifying the Probability of Backtest Overfitting: A Step-by-Step Guide with Python Code and Visual aids | by Francesco Landolfi | Python in Plain English


And then you will realise that you are just overtraining, no matter what criterion you use :

away to push the standard deviation closer to the mathematical expectation or the slope of the regression line or profit maximisation or Sharpe or....


What to do:

You are now just fitting a nice curve through many iterations, the error of multiple testing shows that even at random it is possible to build a TS that will show a nice curve both on the test and on the traine.


And it is necessary

1) Develop a system of simulations, confidence intervals and take the curvature as a result of not one calculation of trading TS as you have, but for example 50 simulations of TS in different environments, the average of these 50 simulations to take as a result of the fitness function that should be maximised/minimised.


2) During the search for the best curve ( from point 1 ) by the optimisation algorithm, each iteration should be correlated for multiple testing.

The problem of multiple testing in practice / Habr (habr.com)


That's how it is...

I've heard this before. All that's beautiful is retraining. Yeah, sure, it's a fitness function and we're looking for the wrong thing, the search criteria are wrong too. Fitness function... I don't have a neural net, if anything. The problems are understandable. It's just a problem of limited sampling, I'll tell you the short of it, you're just not paying attention to what I'm saying. You threw a hundred and five hundred articles at me, as if we have time to sit and read to prove anything to you. What you offer is understandable, but to put it all together in a product and give people you will count it all until retirement and not the fact that you will get your coveted grail.... Resources are limited and time is limited, personally for me, if you have more time for God's sake go deeper. I have heard a lot of things, I have not read articles, but these problems are obvious and without articles, for a person who thinks.

 
Evgeniy Ilin #:

I've heard that before. All that's beautiful is retraining. Yeah, sure, fitness functions and we're looking for the wrong thing, search criteria are wrong too. Fitness function..... I don't have a neural net, if anything. The problems are understandable. It's just a problem of limited sampling, I'll tell you the short of it, you're just not paying attention to what I'm saying. You threw a hundred and five hundred articles at me, as if we have time to sit and read to prove anything to you. What you offer is understandable, but to put it all together in a product and give people you will count it all until retirement and not the fact that you will get your coveted grail.... Resources are limited and time is limited, personally for me, if you have more time for God's sake go deeper. I have heard a lot of things, I have not read articles, but these problems are obvious and without articles, for a person who thinks.

Judging by your answer, you don't understand a damn thing...
I wasted my time. I won't do it again.
 
mytarmailS #:
Judging by your answer, you don't understand a damn thing....
I wasted my time. I won't do it again

Well, I can see you don't understand a damn thing, so who's right? Your judgement is just your judgement and nothing more. For example, I see that you have read clever articles and are pouring here links pretending to be a mega trader, but in fact no one will read it. I have seen your kind, you know a lot of clever words, but there is no use. You need to understand and derive formulas, research, have your own experience and your own position. I have been involved in crypto and sports betting and know everything, I have nothing to do to read your articles. Everything I need I deduce myself, take a notebook and write formulas.

 
mytarmailS #:

You have to

1) Develop a system of simulations, confidence intervals and take the curve as a result of not one calculation of trading TS as you have, but for example 50 simulations of TS in different environments, the average of these 50 simulations to take as a result of the fitness function, which should be maximised/minimised.


2) During the search for the best curve ( from point 1 ) by the optimisation algorithm, each iteration should be correlated for multiple testing.

Are there any examples when anyone has used this approach and brought it to a practical result? Question without mockery, really interesting.

 
Kristian Kafarov #:

Are there any examples when anyone has used this approach and brought it to a practical result? The question is without mockery, really interesting.

I have and I do.
And not only me, all these approaches are widely known and used in science, medicine, etc. (it is a common world practice).

If you want figures regarding the market, let's say that what the author of the article suggests is a usual primitive adjustment to history (retraining) which works on new data almost never ...
In a normal language this all is written in 15 lines of code, but the author spends months on it because as he says "time is precious to him" and proudly calls this useless nonsense a "product".

And what I tried to cover works at least ten times better than primitive fitting.
 
mytarmailS #:
I have and am applying it.

It would be interesting to see concrete examples. It is clear that many people just apply (albeit successfully) and keep silent. But someone should have detailed descriptions of what they did, what they got, and how they traded further.