Neural network in the form of a script - page 11
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
" change in the number of neurons in the layers"
Number of neurons in layers
1. the same:
2. different:
a) increase in number from layer to layer;
b) decrease from one layer to another.
c) increase (decrease) ratio?
The product of the number of inputs by the size of the training sample (number of patterns) should be equal to the square of the NS weights.
I'm a little scared by this figure. Is this even a minimum number for normal operation or an optimal one?
In other words, if I give a vector of 20 values at the input and have the total number of vectors of at least 50000, it means that the number of weights in the net must be no less than Sqrt(20*50000)=1000? Am I right?
A slightly rephrased question from Andy_Kon.
I've seen examples of networks that are organised according to the wave principle. e.g. 20-50-10-1 or 20-40-1. That is, there is an expansion of the network in the middle. (number of neurons of the hidden layer is several times larger than input layer).
From tests of my meshes I came to conclusion that the more hidden layers of neurons, learning goes more smoothly and the result is kind of smeared out on output. If for example 20-20-1, then the network finds solution more sharply and produces sharp values at the outputs.
If for example any theoretical and practical principles of optimal number of neurons in hidden layers depending on the number of input neurons.
Thank you.
" change in the number of neurons in the layers"
Number of neurons in layers
1. the same:
2. different:
a) increase in number from layer to layer;
b) decrease from one layer to another.
c) increase (decrease) ratio?
Well, I gave a link to a theorem, according to which one hidden layer is enough.
That's why all enumerated problems naturally disappear. Another thing is if you decided to decrease dimensionality of inputs using nonlinear principal components method or use hybrid NS with competitive layer... but then the questions have to be appropriate as well.
I was a bit intimidated by this figure. Is this even the minimum number for normal operation or the optimum?
That is, if I take a vector of 20 values and have the total number of vectors of at least 50000, it means that the entire network of weights should contain not less than Sqrt(20*50000)=1000? Is it right?
Correct.
Here is a link to the book by Ezhov and Shumsky "Neurocomputing" where this question is studied superficially (intelligibly).
What is the dependence of the dimensionality and "layering" of the network on the number of patterns (patters)?
1. The hidden layer must be larger than the input layer by at least 1 element, otherwise the information on the hidden layer will be compressed, which does not help the result at all.
2. Consider the number of parameters to be adjusted. If the number of parameters exceeds the number of patterns, you risk having an over-trained network. There must be more patterns. You can reduce the size of the network while the network is learning.
A slightly rephrased question from Andy_Kon.
I've seen examples of networks that are organised according to the wave principle. e.g. 20-50-10-1 or 20-40-1. That is, there is an expansion of the network in the middle. (number of neurons of the hidden layer is several times larger than input layer).
From tests of my meshes I came to conclusion that the more hidden layers of neurons, learning goes more smoothly and the result is kind of smeared out on output. If for example 20-20-1, then the network finds solution more sharply and produces sharp values at the outputs.
If for example any theoretical and practical principles of optimal number of neurons in hidden layers depending on the number of input neurons.
Thank you.
Better yet, 20-21-1
Or better still, 20-21-1
By the way, I also noticed that the hidden layer does not make the number of neurons a multiple of the input layer. >> Why?
Well, I gave a link to the theorem that says that one hidden layer is enough.
So all the above problems naturally disappear. Another thing is if you decided to decrease dimensionality of inputs using nonlinear principal components method or use hybrid NS with competitive layer... but then the questions must be appropriate.
Nevertheless, many problems are solved with a 5-layer perseptron, the existence of the theorem does not mean that a 3-layer perseptron is a panacea.
Which is better -- using a 5-6-6-2 network or a 3-layer replacement of 5-25-2 ? Such a large number might well work for proper nonlinearity.
By the way, do you know the most convergent architecture for XORa?
4 neurons middle -- sigmoidal
By the way, I also noticed that the number of neurons in the hidden layer is not a multiple of the input layer. >> Why?
In my experience, the closer to the input, the better. 21 is optimal for 20 inputs.
In my practice, the closer to the input, the better, 21 -- optimum for 20 inputs
Hmmm... is there any way to summarise this optimum. And about the 5 and 3 ply ones I wonder too. Where's the theory?