Discussion of article "How to use ONNX models in MQL5" - page 2
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
In the 1st comment of the video, note that you need to explicitly specify tensorflow version 2.10.0
Thanks, I mean
There is no point in experimenting there, because such a forecast does not differ from a naive one (the value of the previous closing price is taken as a forecast). In this case, you really get almost the smallest learning error (RMS), which says nothing about the predictive ability of the model. Rather, it is an educational example on ONNX that even complex architecture can be easily transferred to the terminal. I don't know what the authors of that article on research of neural network architectures for time series forecasting were smoking :) here either estimation is needed correctly, or classification instead of regression.
Please explain the example from the picture where the other 20% of test candles are involved. Is it a forward? Why does the forecast almost completely coincide with the actual prices (judging by the picture, it looks like two candlesticks, only one is shifted by a bar), but the tester is a bit sad?
Please explain the example from the picture where the other 20% of the test candles are involved. Is it a forward? Why does the forecast almost completely coincide with the actual prices (judging by the picture, it looks like two candlesticks, only one is shifted by a bar), but in the tester there is some sadness?
You answered your own question, that you get a lagging MA as a forecast (usually with a single shift back). This is a well-known trick when training through standard metrics, they are minimised with this result.
Above I added a link where to read about other metrics.https://stackoverflow.com/questions/65316205/implementation-os-nmae-as-metric-with-keras
You answered your own question, that you get a lagged MA as a prediction (usually with a single backward shift). This is a well-known trick when training through standard metrics, they are minimised with this result.
Above I added a link where to read about other metrics.An article is needed on this topic.
An article is needed on this topic.
Apparently metrics from the energy industry don't fit either :) i don't know a good solution, so i switched to classification.
In google for best forecast metric, MAPE ( Mean absolute percentage error) comes up.
To explicitly use metrics in optimising keras models, they should be specified explicitly as a loss function, not as in #13.
An implementation of over 100 regression metrics in Python can be found here (Akaike Information Criterion, Aitchison distance, Amemiya's Adjusted R-squared, Amemiya's Prediction Criterion, Bias as and given by Gupta, Bayesian Information Criterion, Brier score, Pearson correlation coefficient, Centered root-mean-square (RMS) difference, Decomposed MSE developed by Kobayashi and Salam...).
Example with MAPE metric optimisation (keras should be used instead of numpy math functions, because tensors).
# example from https://github.com/AtrCheema/SeqMetrics/blob/master/SeqMetrics/_rgr.py
# Mean Absolute Percentage Error (MAPE)
# The MAPE is often used when the quantity to predict is known to remain way above zero_.
# It is useful when the size or size of a prediction variable is significant in evaluating the accuracy of a prediction_.
# It has advantages f scale-independency and interpretability.
# However, it has the significant disadvantage that it produces infinite or undefined values for zero or close-to-zero actual values_.
# .. _zero:
# https://doi.org/10.1016/j.neucom.2015.12.114 // Mean Absolute Percentage Error for regression models
# .. _prediction:
# https://doi.org/10.1088/1742-6596/930/1/012002 // Forecasting Error Calculation with Mean Absolute Deviation and Mean Absolute Percentage Error
# .. _values:
# https://doi.org/10.1016/j.ijforecast.2015.12.003 // A new metric of absolute percentage error for intermittent demand forecasts
from keras import backend as K
def MAPE_metric(y_true, y_pred):
return (K.mean(K.abs((y_true - y_pred) / (y_true+1e-8)) * 100))
#define the model
model = Sequential()
model.add(Conv1D(filters=256, kernel_size=2,activation='relu',padding = 'same',input_shape=(120,1)))
model.add(MaxPooling1D(pool_size=2))
model.add(LSTM(100, return_sequences = True))
model.add(Dropout(0.3))
model.add(LSTM(100, return_sequences = False))
model.add(Dropout(0.3))
model.add(Dense(units=1, activation = 'sigmoid'))
model.compile(optimizer='adam', loss=MAPE_metric , metrics = [MAPE_metric])
how its optimisation looks like
predictive graph:
However, using MAPE in the optimisation here also produced a shift.
Perhaps we need to consider more complex directions of optimisations in the form of products of metrics.
The topic requires detailed research.
Google for best forecast metric shows MAPE ( Mean absolute percentage error)
To explicitly use metrics in optimising keras models, they should be specified explicitly as a loss function, not as in #13.
Implementation of more than 100 regression metrics in Python can be found here (Akaike Information Criterion, Aitchison distance, Amemiya's Adjusted R-squared, Amemiya's Prediction Criterion, Bias as and given by Gupta, Bayesian Information Criterion, Brier score, Pearson correlation coefficient, Centered root-mean-square (RMS) difference, Decomposed MSE developed by Kobayashi and Salam...).
However, when MAPE was used in the optimisation process, a shift was also obtained here.
Perhaps we need to consider more complex directions of optimisations in the form of products of metrics.
The topic requires a detailed study.
IMHO, the problem is always not in the metric, but in the structure of the data selected for forecasting - in this case, the most "straightforward" set of samples was taken for the example. Try, for example, to sample vectors through the embedding space (take samples with a step where the autocorrelation of the series disappears) - you will get a more similar forecast to the real one - it means not only the absence of MA-like delay, but also that it will sometimes be very wrong ;-).
If you bite out and glue together parts of the series without news background, the result should be acceptable.
IMHO, the problem is always not in the metric, but in the structure of the data selected for forecasting - in this case the most "straightforward" set of samples was taken for the example. Try, for example, sampling vectors through the embedding space (take samples with a step where the autocorrelation of the series disappears) - you will get a forecast more similar to the real forecast - this means not only the absence of MA-like delay, but also that it will sometimes be very wrong ;-).
This requires more complex data preprocessing processes and a separate treatment of the topic. It seems that a lot here depends on the nature of the relationships between the data (markets have complex ones).
As far as I understand, if there is no new information (or its role is insignificant in the sense of linear transformations Optimum Signal Processing, 1.4 Correlation Cancelling 1.5 Gram-Schmidt Orthogonalisation), then the best forecast will be the past value (but if it is correctly taken into account, it should not be wrong much, though it needs experiments).
On metrics ("directions" of optimisation of network weights) I think it is necessary to understand (for financial time series), the question requires a separate article.
We invite everyone to share their experience.
Any machine learning project consists of three stages:
Using ONNX models in µl solves the implementation issue. It is true that not all models and not quite easy.
Training and optimisation of models is solved by a separate process in Python.
But of all the above, the first stage is the most time-consuming, highly creative and the most important. And it is impossible to realise it on µl. We don't consider primitive scaling as preprocessing. And the folk wisdom says: "Garbage in - rubbish out". There is too much to be additionally developed and implemented in MCL for full use of MO only on MCL. You can't embrace the immense, especially as it is constantly expanding.
Therefore, to execute a preprocess, either make it in another language (whoever has mastered R/Python/Julia, etc.) or try to convert it to ONNX.
The benefit of implementing ONNX so far is that we will learn how to convert, create, simplify and optimise ONNX models. It may be useful in the future.