Discussing the article: "Neural Networks in Trading: Probabilistic Time Series Forecasting (Encoder)"

New comment
 

Check out the new article: Neural Networks in Trading: Probabilistic Time Series Forecasting (Encoder).

We invite you to explore a new approach that combines classical methods and modern neural networks for time series analysis. The article provides a detailed explanation of the architecture and operating principles of the K²VAE model.

The main advantage of K²VAE is not simply generating a forecast, but rather constructing a probability distribution of the system's future states. Unlike traditional models, which are limited to a single most likely scenario, this approach yields a range of possible outcomes. Moreover, the width of this range depends on the model's degree of confidence in the current state. This makes the framework particularly useful in fields where it is important to account for risks and uncertainty — for example, in financial forecasting, logistics, or the management of technical systems.

To understand how this flexibility and adaptability are achieved, let us examine the model's overall architecture. The K²VAE architecture can be broadly divided into three major components: Patching, the Encoder, and the Decoder, each of which performs its own role while being closely interconnected with the others.

  1. Patching prepares the input data and maps it to a latent representation.
  2. The Encoder is responsible for extracting the hidden state Z from the observed time series X. Unlike standard VAE models, this model uses a complex architecture that includes:
    • KoopmanNet, a trainable counterpart of the Koopman operator that predicts the evolution of hidden features as a linear system;
    • Attention Module, which analyzes the differences between reconstructed and actual values, making it possible to identify moments when the model diverges from reality;
    • KalmanNet, a hybrid neural-network implementation of the Kalman filter that forms an uncertainty covariance matrix based on attention control signals;
    • The VAE mechanism, which samples future tokens based on parameters provided by KalmanNet and KoopmanNet.
  3. The Decoder converts hidden variables back into observed variables, reconstructing the predicted values of the time series. At the same time, to preserve the probabilistic nature of the model, the Decoder is also implemented as a trainable neural network structure with two outputs: the mean and the variance. This allows us to fully model the distribution P(Y|Z) and account for the uncertainty in the forecast.

The author’s visualization of the K²VAE framework is shown below.


Author: Dmitriy Gizlyk

New comment