Machine learning in trading: theory, models, practice and algo-trading - page 3722

[Deleted]  
Aleksey Nikolayev #:

Imho, the usual situation when advertising hype is replaced by the usual routine everyday life. If someone fishes through MOE, it is unlikely that they will share their knowledge of fishing spots.

Incidentally, a similar thing is happening with AI right now. The howling about the soon AI chat rooms taking over the power has almost completely disappeared, the soon AGI/ASI is much less often. At most, they periodically roll out the next models that have gained another trillion per cent on the next murky benchmarks. And Bezos are quietly whining about the AI-bubble, as this holiday is mostly at their expense.

I'll sketch an auto-causality indicator on the article later, I'll try to do something with it :)
 
Aleksey Nikolayev #:

Imho, the usual situation when advertising hype is replaced by the usual routine everyday life. If someone fishes through MOE, it is unlikely that they will share their knowledge of fishing spots.

Incidentally, a similar thing is happening with AI right now. The howling about the soon AI chat rooms taking over the power has almost completely disappeared, the soon AGI/ASI is much less often. At most, they periodically roll out the next models that have gained another trillion per cent on the next murky benchmarks. And all Bezos slowly whine about the AI-bubble, as this holiday is mostly at their expense.

Again, it didn't take off.

A bit of history

https://www.mql5.com/en/articles/525

http://www.onix-trade.net/forum/index.php?search/47183954/& page=4

 
Maxim Dmitrievsky #:
I'll sketch an auto-causality indicator later on the article, I'll try to do something with it :)
Something should be done for sure) But I'm too lazy to write anything about it for some reason)
 
Leonid is handsome, he has been in the news recently, he also produces Buzova as I know.....
 
Aleksey Nikolayev #:


    Bottom line

    A joint probabilistic and causal machine learning approach for non-stationary time series:

    • Adapts probabilistic models to account for dynamic and changing characteristics,

    • extracts causal relationships, providing explainability and insight into the internal mechanics,

    • uses algorithms to generate informative features and account for the stochastic nature of the data,

    • improves forecast accuracy and model robustness to noise and structural shifts,

    which makes it possible to apply such methods to complex real-world problems with dynamic and noisy time series.

    From my vantage point, everything you have written in this and previous posts does not solve the problem of non-stationarity - it is an increasingly accurate and contrived modelling of history.

    Why is this relevant to the future?

    Modelling... Modelling, and then running it outside of the learning period, is often highly incorrect. And even if correct, it still doesn't predict. Using the example of favourite trees: fitted, typed trees, start shifting, and there's an old tree growing, but it's associated with a different teacher value. The non-stationarity of the relationship between predictors and teacher, which is likely or regular, does not arise from the non-stationarity of the original time series.Therefore, any tricks in modelling the non-stationarity of the original time series are meaningless.

     
    СанСаныч Фоменко #:

    From my vantage point, everything you've written in this and previous posts doesn't solve the non-stationarity problem - it's an increasingly accurate and contrived modelling of history.

    Why is this relevant to the future?

    Modelling... Modelling, and then running it outside of the learning period, is often highly incorrect. And even if correct, it still doesn't predict. Using the example of favourite trees: fitted, typed trees, start shifting, and there's an old tree growing, but it's associated with a different teacher value. The non-stationarity of the relationship between predictors and teacher, which is likely or regular, does not arise from the non-stationarity of the original time series.Therefore, any tricks in modelling the non-stationarity of the original time series are meaningless.

    Nothing solves the problem of nonstationarity in its entire breadth. Only certain estimates are possible.

    1) Stationarity/nonstationarity is not a property of specific series. It is a property of the models (random processes) that are supposed to be used to analyse these series. One and the same series may well be considered simultaneously in the framework of stationary and non-stationary models.

    2) Nonstationarity is fundamentally unobservable. One can figuratively imagine all nonstationary random processes as an infinite ocean in which there is a tiny island of stationarity. In this analogy, the nonstationarity accessible to study will be only a small coastal shoal surrounding the island of stationarity. In fact, nonstationarity that is not somehow reducible to stationarity is inaccessible to our analysis. Obviously, such reducible nonstationarity is also infinitesimal in volume in comparison with the irreducible one.

     
    Aleksey Nikolayev #:

    Nothing solves the problem of nonstationarity in all its breadth. Only some estimates are possible.

    1) Stationarity/nonstationarity is not a property of specific series. It is a property of the models (random processes) that are supposed to be used to analyse these series. One and the same series may well be considered simultaneously in the framework of stationary and non-stationary models.

    2) Nonstationarity is fundamentally unobservable. One can figuratively imagine all nonstationary random processes as an infinite ocean in which there is a tiny island of stationarity. In this analogy, the nonstationarity accessible to study will be only a small coastal shoal surrounding the island of stationarity. In fact, nonstationarity that is not somehow reducible to stationarity is inaccessible to our analysis. Obviously, such reducible nonstationarity is also infinitesimal in volume in comparison with the irreducible one.

    Unfortunately, it is the nonstationarity of the series that kills the result of any strategy over time. Doesn't it?
     
    Aleksey Nikolayev #:

    Nothing solves the problem of nonstationarity in all its breadth. Only some estimates are possible.

    1) Stationarity/nonstationarity is not a property of specific series. It is a property of the models (random processes) that are supposed to be used to analyse these series. One and the same series may well be considered simultaneously in the framework of stationary and non-stationary models.

    2) Nonstationarity is fundamentally unobservable. One can figuratively imagine all nonstationary random processes as an infinite ocean in which there is a tiny island of stationarity. In this analogy, the nonstationarity accessible to study will be only a small coastal shoal surrounding the island of stationarity. In fact, nonstationarity that is not somehow reducible to stationarity is inaccessible to our analysis. Obviously, such reducible nonstationarity is also infinitesimal in volume in comparison with the irreducible one.

    Somehow everything is black-black and hopeless.....

    I agree that the problem of nonstationarity as such most likely cannot be solved today.

    But it should be solved in parts.

    Predictors.

    What is the purpose of preprocessing and what are the quality criteria?

    Linking the predictors to the target.

    Has anyone made a histogram of this relationship? What are the criteria for a "good" linkage?

    Probability of class prediction.

    Has anyone plotted histograms of these probabilities? I have, and I've plotted and got some surprising and inexplicable results.

    Namely.

    The original Long-Short teacher is divided into two - Long-Out, Short-Out. Seemingly mirror target variables. But. The histogram of the probability of predicting one target variable something like a bell, and the other something like a trough. Why? Which is better and what is the impact.

    The usual partitioning approach there is no room here for simply making the model more complex. if we take a more complex model it should solve a specific identified problem.

    I have not seen any publications that tell you to do this and that and give selection criteria. The usual approach is to change, evaluate the whole steamroller, and then change and compare the evaluations of the whole steamroller. This is not a solution to specific identified nonstationarity problems, by the way, unlike the approach of Garch.

    [Deleted]  
    СанСаныч Фоменко #:

    I have not seen any publications that tell you to do this and that and specify the selection criteria. The usual approach: change and evaluate the whole steamer, and then change and compare evaluations of all steamers. This is not a solution to specific identified nonstationarity problems, by the way, unlike the approach of Garch.

    You need to do pattern matching and see how many patterns you have in the data and how often they repeat. That will answer all your questions. That is, just look at the data. If they don't repeat (and they don't), then there is nothing to generalise about and it's not an IO problem. That leaves the fascinating kurva-zafits.

    Non-stationarity can be defined through increasing number of features with increasing number of observations, for neat zaphytes. In fore, the number of features always increases with increasing history.
     
    Maxim Dmitrievsky #:
    You need to do pattern matching and see how many patterns you have in your data and how often they repeat. That will answer all the questions. That is, just look at the data. If they don't repeat (and they don't), then there is nothing to generalise about and it's not an IO problem. That leaves the fascinating kurva-zafits.

    Non-stationarity can be defined through increasing number of features with increasing number of observations, for neat zaphytes. In fore, the number of features always increases with increasing history.

    What do you mean by "patterns"?

    In wood patterns are wood and they don't repeat. But there is another observation over the rand forest.

    When sampling 1500 bwr the number is sufficiently 70 trees, after that the error stabilises. Another interesting thing is that if we take more than 1500 bvr, for example 5000, then 70 trees are still enough. But it is not clear whether these are the same trees or different ones? I suspect that these are different trees, i.e. "non-stationarity" of trees is observed, which is why it does not work when running a window outside the sample.