Discussion of article "Neural networks made easy (Part 29): Advantage Actor-Critic algorithm"
Hi Dmitriy
many thanks for the exhaustive and very instructive series of articles. Really well done.
Just a question: after downloading all the code from the attachment of your last article (#29), I cannot compile because of missing CBufferDouble class definition, which I suppose should be inside
NeuroNet_DNG\NeuroNet.mqh
but there isn't.
Am I missing something?
Thanks!
Best regards
Paolo
Hi Dmitriy
many thanks for the exhaustive and very instructive series of articles. Really well done.
Just a question: after downloading all the code from the attachment of your last article (#29), I cannot compile because of missing CBufferDouble class definition, which I suppose should be inside
NeuroNet_DNG\NeuroNet.mqh
but there isn't.
Am I missing something?
Thanks!
Best regards
Paolo
Hi, in last article I have change CBufferDouble to CBufferFloat. Its help run library at GPU without type double.
Hi Dmitriy,
Great series, I´m a big fan of this work. I also tried compiling the Reinforce EA and saw it also needed the aunto-encoder (of course) so I added the last version included (from post 22) VAE.mqh, however for some reason it cant find the Normal.mqh definitions:
I'm sure I've made something wrong, hope you could help.
Cheers!
Hi Dmitriy,
Great series, I´m a big fan of this work. I also tried compiling the Reinforce EA and saw it also needed the aunto-encoder (of course) so I added the last version included (from post 22) VAE.mqh, however for some reason it cant find the Normal.mqh definitions:
I'm sure I've made something wrong, hope you could help.
Cheers!
Hi, Load last version at this article https://www.mql5.com/ru/articles/11804
- www.mql5.com
Thank you Dmitriy for the quick response and providing your help and valuable time, however i still got the same result.
Seemingly FQF-learning calls for FQF.mqh
Which in turn calls for NeuroNet...
And of course this last one calls for VAE.mqh
And the only version I could find is the one from post 22...
Using that version ends up on the VAE not finding reference to Normal.mqh functions
Would it be my Editor version?
Thank you.
...Well for some reason the Normal lib is unreachable on VAE.mqh if it is called from NeuroNet, I really dont know why (i tried on 2 different builds)...
So I solved it by adding the call to Normal directly on VAE and Neuronet but I had to get rid of Math space on the FQF:
weird... but it worked:
The initialization failed due to no EURUSD_PERIOD_H1_REINFORCE.nnw when executing the following statements
if(!Actor.Load(ACTOR + ".nnw", dError, temp1, temp2, dtStudied, false) ||
!Critic.Load(CRITIC + ".nnw", dError, temp1, temp2, dtStudied, false))
return INIT_FAILED;
How to solve this issue? Thanks.
Another solution for a warning "... hidden method calling ..."
In Line 327 of Actor_Critic.mq5:
I am getting the warning "deprecated behavior, hidden method calling will be disabled in a future MQL compiler version":
This refers to the call of "Maximum(0, 3)", which must be changed to:
So in this case we have to add "CArrayFloat::" to specify the meant method. The Maximum() method is overwritten by the class CBufferFloat, but this one has no parameters.
Although the call should be unambiguous because it has two parameters, the compiler wants us to be conscious ;-)
The initialization failed due to no EURUSD_PERIOD_H1_REINFORCE.nnw when executing the following statements
if(!Actor.Load(ACTOR + ".nnw", dError, temp1, temp2, dtStudied, false) ||
!Critic.Load(CRITIC + ".nnw", dError, temp1, temp2, dtStudied, false))
return INIT_FAILED;
How to solve this issue? Thanks.
In these lines the network structure that should be trained is loaded. You have to build the network and save it in the named file before starting this EA. You can use e.g. the model building tool in Article No 23
- www.mql5.com
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
New article Neural networks made easy (Part 29): Advantage Actor-Critic algorithm has been published:
In the previous articles of this series, we have seen two reinforced learning algorithms. Each of them has its own advantages and disadvantages. As often happens in such cases, next comes the idea to combine both methods into an algorithm, using the best of the two. This would compensate for the shortcomings of each of them. One of such methods will be discussed in this article.
The advantage of the additional training of the models from previous articles is that we can use test EAs from the previous article to check the results of their training. This is what I used. After training the model, I took the additionally trained policy model and launched the "REINFORCE-test.mq5" EA in the strategy tester using the mentioned model. Its algorithm was described in the previous article. Its full code can be found in the attachment.
Below is a graph of the EA's balance during testing. You can see that the balance was increasing evenly during testing. Note that the model was tested on data outside the training sample. This indicates the consistency of the approach to building a trading system. To test only the model, all operations were performed with a fixed minimum lot without using stop loss and take profit. It is highly not recommended to use such an EA for real trading. It only demonstrates the work of the trained model.
On the price chart, you can see how quickly losing trades are closed and profitable positions are held for some time. All operations are performed at the opening of a new candlestick. You can also notice several trading operations performed almost at the opening of reversal (fractal) candlesticks.
Author: Dmitriy Gizlyk