Discussing the article: "Training Neural Networks on Oscillators Without Look-Ahead Bias"

 

Check out the new article: Training Neural Networks on Oscillators Without Look-Ahead Bias.

The article describes an approach to trade labeling using oscillators for machine learning models. This eliminates look-ahead bias. It has been shown that this type of labeling does not lead to model overfitting, and the strategies continue to perform well over the long term.

For traders who use machine learning, data preparation is an important part of experiments in developing trading systems. Indicators or price increments are typically fed into the model. Future price increments — that is, how the price chart behaved in the future — are often used as labels.

This method of trade labeling has a particularly problematic feature that leads to poor model performance on new, non-stationary data after training on historical data. The ineffectiveness of this labeling method stems from “look-ahead bias,” which is defined as a model's distortion or bias relative to true patterns. Look-ahead bias in the analysis of financial time series occurs when a model or strategy uses information that was not available at the time the decision was made.

The consequences of this are:

  • Inflated and unrealistic results on historical data (backtest). The model demonstrates nearly perfect profitability because it essentially “peeked at the answers” during training.
  • A sharp drop in performance on out-of-sample (OOS) data and in live trading. When a model encounters new data where the future is unknown, it cannot identify the ideal patterns to which it has “become accustomed,” and its predictions become useless.
  • Low model robustness (stability). The strategy turns out to be overfitted to historical data and unable to adapt to changing market volatility and regimes.
Researchers often underestimate this problem as they search through features that would both match the labels as closely as possible and predict them. However, such a match can never be achieved because of the random and idealized nature of the labels themselves.


Author: dmitrievsky

 

Isn’t the promotional image for the article somewhat misleading? The markup in it clearly doesn’t follow the rules set out in the text or code.

Why is `close_data` passed to the `calculate_labels_rsi` function? Wouldn’t it be more correct to write the loop in it like this:

    for i in range(len(rsi_data)):

?

[Deleted]  
Stanislav Korotky #:

Isn’t the promotional image for the article a bit misleading? The markup in it clearly doesn’t follow the rules set out in the text or code.

Why is `close_data` passed to the `calculate_labels_rsi` function? Wouldn’t it be better to write the loop in it like this:

?

"AI" always plots it this way, apparently.

Other conditions where price data is required can be added to the code. Originally, this was a function for calculating divergences between prices and oscillators.

 
Stanislav Korotky #:

Isn’t the promotional image for the article a bit misleading? The markup in it clearly doesn’t follow the rules set out in the text or code.

Why is `close_data` passed to the `calculate_labels_rsi` function? Wouldn’t it be better to write the loop in that function like this:

?

The author has nothing to do with this
 
Rashid Umarov #:
The author has nothing to do with this
I have no complaints about the author. But that does not alter the essence of the problem. As the editor-in-charge, do you not find this misleading (due to the use of an inappropriate image)?
 
Stanislav Korotky #:
I have no complaints about the author. But that doesn’t change the crux of the matter. As the editor-in-charge, don’t you think this is misleading (due to the use of an inappropriate image)?

Speaking of which, take a look at the preview

Researchers often underestimate this problem by trying to identify features that would not only best match the labels but also predict them. However, such a match can never be achieved due to the random and idealised nature of the labels themselves.


 
Rashid Umarov #:

That’s exactly what we’re talking about – take a look at the preview

The wording here doesn’t quite add up. The image shows incorrect labels, not idealised ones.

We need more detail on random labels. No machine learning model will work with random labels. Presumably, something else was meant here, rather than labels being applied haphazardly.

[Deleted]  
Stanislav Korotky #:

Here, the text contradicts logic. The image shows incorrect labels, not idealised ones.

We need more detail about random labels. No machine learning model will work with random labels. Presumably, something else was meant here, rather than labels being assigned haphazardly.

On new data, the idealised labels will turn into incorrect ones :) We could discuss this endlessly.

Random labels depend on random price changes that occurred in the future. This is explained in the article.

A similar question for you: what do you mean by ‘no ML will work with random labels’? Perhaps you meant to say something else? :)
 
Maxim Dmitrievsky #:

With new data, idealised models will turn out to be incorrect :) We could discuss this endlessly.

Random labels depend on random price changes that occurred in the future. The article mentions this.

A similar question for you: what do you mean by “no ML model will work with random labels”? Perhaps you meant to say something else? :)

The wording in this section is slightly ambiguous, but the essence of the approach is clear from the article. I had no complaints about the article itself.

We don’t need to discuss it further; as far as I’m concerned, labels always come from the past, not the future, and if we’re talking about their contribution to the model’s future performance, it’s incorrect to dismiss them all as erroneous. Let’s each stick to our own opinions.

My comment about random labels – it doesn’t actually seem to be my own, but rather a well-known fact in machine learning.