Discussing the article: "Neural Networks in Trading: Transformer with Relative Encoding"

[MetaQuotes]

Check out the new article: Neural Networks in Trading: Transformer with Relative Encoding.

Self-supervised learning can be an effective way to analyze large amounts of unlabeled data. The efficiency is provided by the adaptation of models to the specific features of financial markets, which helps improve the effectiveness of traditional methods. This article introduces an alternative attention mechanism that takes into account the relative dependencies and relationships between inputs.

Transitioning from training models from scratch to pretraining on large sets of unlabeled data, followed by fine-tuning for specific tasks, allows us to achieve high-accuracy forecasting without the need to collect massive volumes of new data. For example, models based on the Transformer architecture, adapted for financial data, can leverage information on asset correlations, temporal dependencies, and other factors to produce more accurate predictions. The implementation of alternative attention mechanisms helps account for key market dependencies, significantly enhancing model performance. This opens new opportunities for developing trading strategies while minimizing manual tuning and reliance on complex rule-based models.

One such alternative attention algorithm was introduced in the paper "Relative Molecule Self-Attention Transformer". The authors proposed a new Self-Attention formula for molecular graphs that meticulously processes various input features to achieve higher accuracy and reliability across many chemical domains. Relative Molecule Attention Transformer (R-MAT) is a pretrained model based on the Transformer architecture. It represents a novel variant of relative Self-Attention that effectively integrates distance and neighborhood information. R-MAT delivers state-of-the-art, competitive performance across a wide range of tasks.

Author: Dmitriy Gizlyk