Discussion of article "Neural networks made easy (Part 12): Dropout"
Note, when used in Fractal_OCL_AttentionMLMH_d instead of Net.feedForward(TempData,12,true);
printf("error in Net feed forward. check topology and input data");
will generate an error. Unlike the Fractal_OCL_AttentionMLMH version where there will be no such error.
Please check the reason.
Note when used in Fractal_OCL_AttentionMLMH_d instead of Net.feedForward(TempData,12,true);
printf("error in Net feed forward. check topology and input data");
will generate an error. Unlike the Fractal_OCL_AttentionMLMH version where there will be no such error.
Please check the reason.
Thanks, I will check it.
Note when used in Fractal_OCL_AttentionMLMH_d instead of Net.feedForward(TempData,12,true);
printf("error in Net feed forward. check topology and input data");
will generate an error. Unlike the Fractal_OCL_AttentionMLMH version, where there will be no such error.
Please check the reason.
Try this version, I had no errors.
Thank you very much for this article. I have already adapted my robots for the new features and they work great.
I take it that nothing works without OCL? Too bad, I'm not a gamer and the card is old....
Dmitry, can you explain a little bit more about the window parameter in the creation of Neuronkey. Am I right to understand that this is the number of neurons sequentially (with a step) that are taken for one-step calculation? To save computing power? |
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
New article Neural networks made easy (Part 12): Dropout has been published:
As the next step in studying neural networks, I suggest considering the methods of increasing convergence during neural network training. There are several such methods. In this article we will consider one of them entitled Dropout.
When training a neural network, a large number of features are fed into each neuron, and it is difficult to evaluate the impact of each separate feature. As a result, the errors of some neurons are smoothed out by the correct values of other neurons, while these errors are accumulated at the neural network output. This causes the training to stop at a certain local minimum with a rather large error. This effect is referred to as the co-adaptation of feature detectors, in which the influence of each feature adjusts to the environment. It would be better to have the opposite effect, when the environment is decomposed into separate features and it is possible to evaluate the influence of each feature separately.
In 2012, a group of scientists from the university of Toronto proposed to randomly exclude some of the neurons from the learning process as a solution to the complex co-adaptation problem [12]. A decrease in the number of features in training increases the importance of each feature, and a constant change in the quantitative and qualitative composition of features reduces the risk of their co-adaptation. This method is called Dropout. Sometimes the application of this method is compared to decision trees: by dropping out some of the neurons, we obtain a new neural network with its own weights at each training iteration. According to the combinatorics rules, such networks have quite a high variability.
Author: Dmitriy Gizlyk