SAMformer offers a solution to the key drawbacks of Transformer models in long-term time series forecasting, such as training complexity and poor generalization on small datasets. Its shallow architecture and sharpness-aware optimization help avoid suboptimal local minima. In this article, we will continue to implement approaches using MQL5 and evaluate their practical value.
Training for all three models was conducted simultaneously. The results of testing the trained Actor policy are presented below. The testing was performed on real historical data for January 2024, with all other training parameters unchanged.
Before examining the results, I would like to mention several points regarding model training. First, SAM optimization inherently smooths the loss landscape. This, in turn, allows us to consider higher learning rates. While in earlier works we primarily used a learning rate of 3.0e-04, in this case we increased it to 1.0e-03.
Second, the use of only a single attention layer reduced the total number of trainable parameters, helping to offset the computational overhead introduced by the additional feed-forward pass required by SAM optimization.
But the results of testing on EURUSD and the result in USD are given in the article. At the same time, the load on the deposit is 1-2%. And nobody wrote that it is a grail.
But the article gives the results of testing on EURUSD and the result in USD. At the same time, the load on the deposit is 1-2%. And no one wrote that it is a grail.
Allow the use of cookies to log in to the MQL5.com website.
Please enable the necessary setting in your browser, otherwise you will not be able to log in.
You are signed in. Two-factor authentication has been disabled. Set it up again and scan a new QR code, or continue with two-factor authentication disabled.
Check out the new article: Neural Networks in Trading: Enhancing Transformer Efficiency by Reducing Sharpness (Final Part).
Training for all three models was conducted simultaneously. The results of testing the trained Actor policy are presented below. The testing was performed on real historical data for January 2024, with all other training parameters unchanged.
Before examining the results, I would like to mention several points regarding model training. First, SAM optimization inherently smooths the loss landscape. This, in turn, allows us to consider higher learning rates. While in earlier works we primarily used a learning rate of 3.0e-04, in this case we increased it to 1.0e-03.
Second, the use of only a single attention layer reduced the total number of trainable parameters, helping to offset the computational overhead introduced by the additional feed-forward pass required by SAM optimization.
Author: Dmitriy Gizlyk