Discussing the article: "Neural Networks in Trading: An Intelligent Forecast Pipeline (Time-MoE)"

[MetaQuotes]

Check out the new article: Neural Networks in Trading: An Intelligent Forecast Pipeline (Time-MoE).

We invite you to explore the modern Time-MoE framework, which has been adapted for time series forecasting tasks. In this article, we will implement the key components of the architecture step by step, providing explanations and practical examples along the way. This approach will allow you not only to understand how the model works, but also to apply those principles to real-world trading scenarios.

Time-MoE is a next-generation decoder-only Transformer designed specifically for time series data. It is based on the principles of sparse learning, modularity, and forecasting at multiple scales. The framework's authors demonstrate how the scalability and flexibility of large models can be extended to the realm of time series without sacrificing computational efficiency. The model architecture they present supports arbitrary lengths of analyzed sequences and forecast horizons. At the same time, the model can process data streams in real time.

The first component of the architecture is point-wise tokenization of the time series (Point-Wise Tokenization). Unlike window-based or aggregated approaches, each time step here is converted into a separate token. This feature can be useful in high-frequency trading, where even a single tick can change the market picture. In the MQL5 environment, tokens can be formed from bars, ticks, and derived indicators, including volatility, volume, and signals from custom strategies.

After tokenization, the data are passed through an embedding layer with SwiGLU activation, which is a hybrid of Swish and a Gated Linear Unit. It allows for smoother and more robust representations of the analyzed information, which is particularly useful in the presence of market noise and unstable trends.

Author: Dmitriy Gizlyk