Discussing the article: "Developing a robot in Python and MQL5 (Part 2): Model selection, creation and training, Python custom tester" - page 2

New comment
 

Still row prices turn out to be the best chips.

I used to be sceptical because of their non-stationarity. But after some manipulations I also started to extract decent models on these features.

So from ignorance comes knowledge, and from knowledge comes ignorance :)

 
Ivan Butko #:
Good motivation when there are results!
And, as I realised, it's not a week ahead, and not a month, but a normal year's worth of work

Thank you very much! Yes, it motivates me a lot! I'll keep on researching) It's night again, I have a cup of coffee and code ideas with me)))))

 
Maxim Dmitrievsky #:

Still row prices turn out to be the best chips.

I used to be sceptical because of their non-stationarity. But after some manipulations I also started to extract decent models on these features.

So from ignorance comes knowledge, and from knowledge comes ignorance :)

Here is a type of such tried, my mother-in-law is a trader with experience of 15+ years, she keeps saying it is necessary to make chips on volumes))) https://www.mql5.com/ru/code/50133

Индикатор Price / Volume
Индикатор Price / Volume
  • www.mql5.com
Одна из простых фич для машинного обучения
 
Yevgeniy Koshtenko #:

This is the kind of thing I tried, my mother-in-law is a trader with 15+ years of experience, she keeps saying you need to make chips on volumes))) https://www.mql5.com/ru/code/50133

Yes, it is true that more often volatility is added (e.g. std indicator), but it doesn't give much. Or increments divided by volatility.

 

Eugene, from your articles I started to study ML in relation to trading, thank you very much for that.

Could you explain the following points.

After the label_data function processes the data, its volume is significantly reduced (we get a random set of bars that satisfy the conditions of the function). Then the data goes through several functions and we divide it into train and test samples. The model is trained on the train sample. After that, the ['labels'] columns are removed from the test sample and we try to predict their values to estimate the model. Is there no concept substitution in the test data? After all, for the tests we use data that has passed the label_data function (i.e. a set of non sequential bars selected in advance by a function that takes future data into account). And then in the tester there is parameter 10, which, as I understand, should be responsible for how many bars to close the deal, but since we have a non sequential set of bars, it is not clear what we get.

The following questions arise: Where am I wrong? Why not all bars >= FORWARD are used for tests? And if we do not use all bars >= FORWARD, how can we choose the bars necessary for prediction without knowing the future?

Thank you.

 
Great work, very interesting, practical and down to earth. Hard to see an article this good with actual examples and not just theory without results. Thank you so much for your work and sharing, I'll be following and looking forward this series.
 
Eric Ruvalcaba #:
Great work, very interesting, practical and down to earth. Hard to see an article this good with actual examples and not just theory without results. Thank you so much for your work and sharing, I'll be following and looking forward this series.

Thanks a lot! Yes, there are still a lot of implementations of ideas ahead, including the expansion of this one with translation into ONNX)

 
Is there any particular reason for using RandomForestClassifier for feature selection and XGBclassifier for model classification?
 

Critical flaws:

  1. Problems in preventing data leakage:
    • The augment_data() function creates serious data leakage problems between the training and test sets
    • Augmentation mixes data from different time periods
  2. Errors in performance evaluation methodology:
    • Model testing does not take into account real market conditions
    • The model is trained on future data and tested on historical data, which is unacceptable
  3. Technical problems in the code:
    • The generate_new_features() function creates features but does not return them (returns the original data)
    • The test_model() uses X_test.iloc[i]['close'] , but 'close' may be missing after transforming the features
  4. Inconsistent data processing:
    • Data is labelled twice in different ways ( markup_data() and label_data() )
    • Clustering results ( cluster ) are not used in further training
  5. Methodological problems in trading strategy:
    • Static exit after 10 bars instead of adaptive strategy
    • No risk management (except for a simple stop-loss)
    • No consideration of transaction costs (other than simple spread)
  6. Ineffective validation:
    • No validation of the model on historical data considering time structure (walk-forward analysis)
    • Cross validation is applied to time series without taking into account time series specificity

Recommendations for improvement:

  1. Eliminate data leakage - strictly segregate data over time
  2. Implement proper walk-forward validation
  3. Implement more realistic testing, taking into account slippage and commissions
  4. Finalise the logic of entering/exiting positions
  5. Use time-series specific methods
12
New comment
close