Discussing the article: "Build Self Optimizing Expert Advisors With MQL5 And Python (Part II): Tuning Deep Neural Networks"

[MetaQuotes]

Check out the new article: Build Self Optimizing Expert Advisors With MQL5 And Python (Part II): Tuning Deep Neural Networks.

Machine learning models come with various adjustable parameters. In this series of articles, we will explore how to customize your AI models to fit your specific market using the SciPy library.

The Nelder-Mead algorithm is a popular choice for noisy, non-differentiable, and non-linear multimodal optimization problems. Named after its inventors John Nelder and Roger Mead, the algorithm was introduced in their 1965 paper titled "A Simplex Method for Function Minimization." It can be used for both univariate and multivariate optimization problems.

The Nelder-Mead algorithm does not rely on derivative information; instead, it is a pattern search optimization algorithm. It requires the user to provide a starting point. Depending on the chosen starting point, the algorithm might get stuck in a deceptive local optimum. Therefore, it can be beneficial to perform the optimization multiple times with different starting points to improve the chances of finding a global optimum.

The algorithm works by using a geometric shape called a simplex. The simplex has one vertex for each input variable plus one additional vertex. The points (vertices) of the simplex are evaluated, and simple rules are used to move the points based on their evaluations. The algorithm has certain stopping conditions, such as reaching the maximum number of iterations or achieving a minimal change in evaluation values. If no improvements are made or if the allowed number of iterations is exceeded, the optimization procedure stops.

Author: Gamuchirai Zororo Ndawana