Discussing the article: "Automatic Differentiation: Exact Gradients Without a Step Size"

Check out the new article: Automatic Differentiation: Exact Gradients Without a Step Size.

We implement a compact automatic differentiation library for MQL5 in forward and reverse modes and use it to supply exact Jacobians to ALGLIB's Levenberg–Marquardt solver. The article details dual numbers, a tape with local partials, validation against closed forms and jax, and timing versus finite differences. You will learn when forward mode fills least‑squares Jacobians faster, where finite differences still win at small parameter counts, and how AD behaves on real price data.

Plenty of useful work in a trading terminal comes down to minimizing something. You fit a decay model to an autocorrelation curve, calibrate a volatility surface, tune a small network's weights, or find the parameter set that best explains a spread. Every one of those is handed to an optimizer, and every serious optimizer wants a derivative.

There are three ways to give it one. You can derive the gradient by hand, which is exact and fast and goes wrong the moment the model changes. You can let the optimizer build one from finite differences, which is what most MQL5 code does and which quietly introduces a number you now have to choose: the step size. Or you can compute the derivative alongside the function itself, by arithmetic rather than approximation. That third option is automatic differentiation, the subject here.

The step size is worth dwelling on, because it is the part that hides. A difference quotient fails in two ways: a large step measures a chord, not a tangent. A small step loses significant digits when subtracting nearly equal doubles. The best step lies in between, depends on the function and the point and the scale of the variable, and nothing tells you when you have picked badly. The optimizer converges either way. It just converges somewhere slightly different.

Automatic differentiation eliminates the step-size choice. Both modes built here produce a derivative exact up to floating point rounding, with no approximation step and no parameter to tune. What follows includes the full library and the measurements behind each claim. It also shows three results that contradict common advice:

The library is six files and about three thousand lines. Its only dependency is ALGLIB, which ships with MetaTrader 5; the Python cross-check is optional.

Author: Hammad Dilber