Discussing the article: "Neural Networks in Trading: Effective Feature Extraction for Accurate Classification (Mantis)"

[MetaQuotes]

Check out the new article: Neural Networks in Trading: Effective Feature Extraction for Accurate Classification (Mantis).

Meet Mantis — a lightweight foundation model for time series classification based on a Transformer architecture, featuring contrastive pre-training and hybrid attention that deliver record-breaking accuracy and scalability.

The Mantis framework demonstrates how well-thought-out architecture and thorough training can transform a set of ideas into a reliable tool. The foundation-model approach presented by the authors of the framework rests on four key pillars: time series tokenization, hybrid attention, adapters for multivariate data, and Self-Supervised contrastive pre-training followed by calibration.

The main idea behind Mantis is to move away from the traditional practice of dividing a time series into fixed windows. Instead, the sequence is divided into a fixed number of patches, which ensures independence from the length of the input sequence and stabilizes the computational cost. For example, series of lengths 1024 and 2048 will be converted into the same number of patches — 32. This approach is critically important for large-scale processing of heterogeneous time series.

The embedding is formed in several stages. First, a convolutional layer with 256 output channels is applied. This layer transforms the time series into a more compact latent representation. Next, each of the 32 patches is aggregated using averaging (mean pooling), resulting in a tensor of dimension (32, 256). Each patch encodes the local characteristics of the time series, including peaks, fluctuations, and the microstructure of price movements.

In parallel, a second data stream — the differential stream — is created. It is based on the first differences between adjacent values in the time series. This transformation helps eliminate long-term trends and amplify signals related to short-term dynamics. It is particularly useful in situations where deviations from a stable level or sharp movements near support and resistance levels are of interest.

Both streams undergo the same processing steps: convolution, averaging, and normalization. The result is two sets of patches, each containing 32 tokens of dimension 256. This provides the model with balanced information about both the signal's shape and its changes over time.

Author: Dmitriy Gizlyk