Soft Actor Critic is yet another reinforcement learning algorithm that we are considering, having looked at a few already that included proximal policy optimization, deep-Q-networks, SARSA, and others. This algorithm though, like some that we have already looked at, uses neural networks, but with some important caveat. The total number of networks used are three, and these are: 2 critic networks and an actor network. The two critic networks make reward forecasts (Q-Values) when inputted with an action and an environment state, and the minimum of the outputs of these 2 networks is used in modulating the loss function used for training the actor network.
Soft Actor Critic (SAC) is an algorithm that optimizes a stochastic policy in an off-policy way, forming a bridge between stochastic policy optimization and DDPG-style approaches. It isn’t a direct successor to TD3 (having been published roughly concurrently), but it incorporates the clipped double-Q trick, and due to the inherent stochasticity...
The Accelerator Oscillator is another Bill Williams Indicator that is designed to keep track of momentum and though common as it is derived from another oscillator that we covered in an earlier article, we take look at how various patterns exhibited by it could be used or even combined in ways that may give one an edge. This indicator specifically aims to track acceleration or deceleration of the momentum, which can be used in providing signals for possible trade entries and exits. Acceleration is different from velocity.
The Accelerator Oscillator is another Bill Williams Indicator that tracks price momentum's acceleration and not just its pace. Although much like the Awesome oscillator we reviewed in a recent article, it seeks to avoid the lagging effects by focusing more on acceleration as opposed to just speed. We examine as always what patterns we can get from this and also what significance each could have in trading via a wizard assembled Expert Advisor.
The article was meant to be on machine learning given the sequence that has been in place with the past pieces; however, I am sticking with a trade indicator because some material for the next machine learning piece is not quite ready. It will be soon, hopefully.
The Market Facilitation Index is another Bill Williams Indicator that is intended to measure the efficiency of price movement in tandem with volume. As always, we look at the various patterns of this indicator within the confines of a wizard assembly signal class, and present a variety of test reports and analyses for the various patterns.
Soft Actor Critic (SAC) is one of the algorithms used in Reinforcement Learning when training a neural network. To recap, reinforcement learning is a budding method of training in machine learning, alongside supervised learning and unsupervised learning.
Soft Actor Critic is a Reinforcement Learning algorithm that we looked at in a previous article, where we also introduced python and ONNX to these series as efficient approaches to training networks. We revisit the algorithm with the aim of exploiting tensors, computational graphs that are often exploited in Python.
The growing complexity of neural network models is driven by our ability to process vast amounts of data. Traditional machine learning struggles with efficiency, while neural networks, exemplified by platforms like DeepSeek, Grok, and ChatGPT, offer powerful solutions.
Replay buffers in Reinforcement Learning are particularly important with off-policy algorithms like DQN or SAC. This then puts the spotlight on the sampling process of this memory-buffer. While default options with SAC, for instance, use random selection from this buffer, Prioritized Experience Replay buffers fine tune this by sampling from the buffer based on a TD-score. We review the importance of Reinforcement Learning, and, as always, examine just this hypothesis (not the cross-validation) in a wizard assembled Expert Advisor.
Bill Williams’ fractal indicator is a pivotal and important indicator amongst the collection habits known for. It primarily identifies reversal points in price action of traded symbols. Based on the concept of fractals, as a repetitive 5-bar pattern often marked as bearish if the middle bar of the 5 has the highest high, or bullish in cases where the middle bar has the lowest low. We look at some of this indicator's patterns that can be utilised by traders, as we have in the past with MQL5 wizard articles.
In supervised learning, we will seek to implement each pattern in a separate neural network. These, as argued in recent articles, are better coded and trained in python than MQL5. The efficiency gains are off the charts. Python also easily allows testing for cross validation following a training session, and thus we will be performing these for each pattern.
Combining the Moving Average (MA) with the Stochastic Oscillator can generate a variety of trading signals. For our testing and exploring purposes, we will only consider the 10 top signal patterns that traders can use when these indicators are combined.
Moving Average and Stochastic Oscillator are very common indicators that some traders may not use a lot because of their lagging nature. In a 3-part ‘miniseries' that considers the 3 main forms of machine learning, we look to see if this bias against these indicators is justified, or they might be holding an edge. We do our examination in wizard assembled Expert Advisors.
From our last article, we tested 10 signal patterns from our 2 indicators (MA & Stochastic Oscillator). Seven were able to forward-walk based on a 1-year test window. However, of these, only 2 did so by placing both long and short trades. This was down to our small test-window, which is why readers are urged to test this on more history before taking it any further.
We are following a thesis here where the three main modes of machine learning can be used together, each in its own ‘phase’.
Moving Average and Stochastic Oscillator are very common indicators whose collective patterns we explored in the prior article, via a supervised learning network, to see which “patterns-would-stick”. We take our analyses from that article, a step further by considering the effects' reinforcement learning, when used with this trained network, would have on performance. Readers should note our testing is over a very limited time window. Nonetheless, we continue to harness the minimal coding requirements afforded by the MQL5 wizard in showcasing this.
In the last article we had introduced DDPG, a reinforcement-learning algorithm and had looked at 3 of its crucial classes as implemented in Python. The replay buffer class, the actor-network class, and the critic-network class. What was not covered was the DDPG-agent class; importing MetaTrader 5 price data to Python; functions for the MA, & Stochastic-Oscillator; a get-pattern function for bringing together data from the two indicators into a binary input vector for the supervised learning network (implemented in the earlier supervised-learning article via MQL5); and finally an environment simulation loop for training the actor & critic networks.
Moving Average and Stochastic Oscillator are very common indicators whose collective patterns we explored in the prior article, via a supervised learning network, to see which “patterns-would-stick”. We take our analyses from that article, a step further by considering the effects' reinforcement learning, when used with this trained network, would have on performance. Readers should note our testing is over a very limited time window. Nonetheless, we continue to harness the minimal coding requirements afforded by the MQL5 wizard in showcasing this.
Allow the use of cookies to log in to the MQL5.com website.
Please enable the necessary setting in your browser, otherwise you will not be able to log in.
You are signed in. Two-factor authentication has been disabled. Set it up again and scan a new QR code, or continue with two-factor authentication disabled.
MQL5 Wizard Techniques you should know (Part 51): Reinforcement Learning with SAC
MQL5 Wizard Techniques you should know (Part 52): Accelerator Oscillator
The pictures always make me happy - one is better than the other!
MQL5 Wizard Techniques you should know (Part 53): Market Facilitation Index
MQL5 Wizard Techniques you should know (Part 54): Reinforcement Learning with hybrid SAC and Tensors
Soft Actor Critic (SAC) is one of the algorithms used in Reinforcement Learning when training a neural network. To recap, reinforcement learning is a budding method of training in machine learning, alongside supervised learning and unsupervised learning.
MQL5 Wizard Techniques you should know (Part 55): SAC with Prioritized Experience Replay
Forum on trading, automated trading systems and testing trading strategies
Chaos Theory by Bill Williams
Sergey Golubev, 2025.03.05 07:31
MQL5 Wizard Techniques you should know (Part 56): Bill Williams Fractals
MQL5 Wizard Techniques you should know (Part 57): Supervised Learning with Moving Average and Stochastic Oscillator
In supervised learning, we will seek to implement each pattern in a separate neural network. These, as argued in recent articles, are better coded and trained in python than MQL5. The efficiency gains are off the charts. Python also easily allows testing for cross validation following a training session, and thus we will be performing these for each pattern.
Combining the Moving Average (MA) with the Stochastic Oscillator can generate a variety of trading signals. For our testing and exploring purposes, we will only consider the 10 top signal patterns that traders can use when these indicators are combined.
MQL5 Wizard Techniques you should know (Part 58): Reinforcement Learning (DDPG) with Moving Average and Stochastic Oscillator Patterns
From our last article, we tested 10 signal patterns from our 2 indicators (MA & Stochastic Oscillator). Seven were able to forward-walk based on a 1-year test window. However, of these, only 2 did so by placing both long and short trades. This was down to our small test-window, which is why readers are urged to test this on more history before taking it any further.
We are following a thesis here where the three main modes of machine learning can be used together, each in its own ‘phase’.
MQL5 Wizard Techniques you should know (Part 59): Reinforcement Learning (DDPG) with Moving Average and Stochastic Oscillator Patterns