Discussing the article: "Working with ONNX Models in MQL5 (Part 1): Decoding the Model File with a Protobuf Parser"

 

Check out the new article: Working with ONNX Models in MQL5 (Part 1): Decoding the Model File with a Protobuf Parser.

We decode ONNX files in pure MQL5 by implementing a Protocol Buffers reader from scratch. We generate a sample network in Python, verify it in Netron, and then parse the same binary to recover graph nodes, connections, weight tensors, and input/output shapes. The result is a MetaTrader 5 program that inspects a trained model's structure before inference, without any external libraries.

An ONNX file is not a bundle of numbers. It is a description of a computation graph, written in a structured format that any framework can read. Inside it sits a graph made of three kinds of things. There are nodes, each naming an operator such as a matrix multiply or an activation, along with the names of the values it reads and the names of the values it produces. There are initializers, which are the trained weights themselves, each carrying a name, a shape, an element type, and a block of raw numbers. And there are the graph ports, the values the model takes in and hands back, each declaring the exact tensor shape the caller must supply or should expect.

What matters for a parser is that the file stores those nodes as a flat list. Nothing in it points at anything else. A node records the names it reads and the names it writes, and that is all. The connections are implied: one node writes a value called "hidden", another reads a value called "hidden", and that shared name is the edge between them. Rebuilding the graph therefore means matching names, and the same rule attaches the weights, since a node reads its weight tensor by name exactly as it reads any other value. See an illustration below of how a graph is stored and how we rebuild it.

How a graph is stored, and how we rebuild it

How a graph is stored, and how we rebuild it

The rule: the node that writes "hidden" is the parent of the node that reads "hidden".

All of that is encoded in Protocol Buffers, a binary format built for compactness rather than readability. A message is a sequence of fields, and every field opens with a tag that packs two things together: which field this is, and how its value is encoded. That second part is what makes a hand-written parser realistic, because it tells a reader how many bytes to consume even when the reader has no idea what the field means. There are only a few encodings in practice. Some values are variable-length integers, where each byte carries seven bits and its top bit says whether another byte follows. Some are length-prefixed blocks, where a count comes first, followed by that many bytes, used for text, for raw weight data, and for nested messages. The rest are fixed at four or eight bytes.

Author: Allan Munene Mutiiria