Neural network in the form of a script - page 10

New comment
 

Please describe the notation in the formula S[j] = Sum(i)(y[i]*w[i,j] - t[j]). As I understand it:

t[j] is the weight of the threshold (it is multiplied by a signal equal to -1)

y[i]*w[i,j] - the input multiplied by its weight

S[j] - the result before the logistic function was applied

What is Sum(i)?

 
sergeev писал (а) >>

Please describe the notation in the formula S[j] = Sum(i)(y[i]*w[i,j] - t[j]). As I understand it:

t[j] is the weight of the threshold (it is multiplied by a signal equal to -1)

y[i]*w[i,j] - input multiplied by its weight

S[j] - the result before the logistic function was applied

What is Sum(i)?

Sum[i] is the sum over i. I don't know how to write formulas here.


t[j] - threshold value of the neuron

y[i]*w[i,j] - the output of the layer multiplied by the binding weight

S[j] is the weighted sum before the logistic function was applied

 
TheXpert писал (а) >>

Sum[i] is the sum over i. I can't write formulas here.

That's basically what I immediately thought was the amount.

--------------

The point is that the threshold is not added to each input value but at the end of the total sum before it is fed to the sigmoid. That is, it is the threshold for the output as a whole and not for each input individually.

The formula is as follows:

S[j]=Sum{y[i]*w[i,j]} - t[j].

It's like Yuri's.

for(i=0;i<NUM_INP;i++) // входной слой
    { 
        sum=0;
        for(j=0;j<NUM_INP;j++) sum+=ipl_W[i,j]*Input[j];
        ipl_Out[i]=Sigmoid(sum+ipl_Threshold[i]);
    }

and in principle I agree with it, because I've seen such a notation in the literature.

 
sergeev писал (а) >>

That's basically what I immediately thought was the amount.

--------------

The point is that the threshold is not added at each input value but at the end of the total sum before it is fed to the sigmoid.

So the formula is

S[j]=Sum{y[i]*w[i,j]} - t[j].

This is how Yuri does it.

and in principle I agree with it, as I have seen such a notation in the literature.



Ugh, of course you're right, I put the brackets wrong.

 

2 TheXpert

I understand from your posts that you are an expert in NS. Could you advise a beginner where to start analyzing in NS to get acquainted with how it works...

And is it a good thing to use feedback in networks? How effective has it been in your practice?

 
sergeev писал (а) >>

2 TheXpert

I understand from your posts that you are an expert in NS. Could you advise a beginner where to start analyzing in NS to get acquainted with the principle of its work...

And is the use of feedback networks also a good thing. How effective has it been in your practice?


Honestly, I don't even know what to say. We had 2 courses on NS at university, we started with basics: basic neuron model, classification of networks, training methods, etc., then perseptron, linear and nonlinear, then Kohonen, Hopfield, Hemming, recurrent, recurrent networks....



About recurrent networks -- not used in practice, IMHO, its advantage and simultaneously disadvantage is that it depends on its previous states, i.e. by definition it is suitable for exchange.

But, again, IMHO, I believe similar results can be obtained by a perseptron without feedback if trained using the sliding window principle. There's also an advantage to this, the sliding window method allows us to assess the robustness/stochasticity (see chaos theory) of the obtained forecast with little blood, which can be of great help when the market is very volatile and the outcome unpredictable.

 
What is the dependence of the dimensionality and "layering" of the network on the number of patterns?
 
Andy_Kon писал (а) >>
What is the dependence of the dimensionality and "layering" of the network on the number of patterns?

Simon Heikin, in his book Neural Networks on page 282, gives a theorem on the universality of NS with ONE hidden layer. Here is the implication:

And what sense you give to the phrase "network dimension" I do not understand. Is it the number of neurons in hidden layers or the number of inputs to NS?

If it's number of inputs then the product of number of inputs by training sample size (number of patterns) must be equal to the square of NS weights.

If it's the number of neurons in the hidden layers, their number is determined by the complexity of the problem and is found experimentally.

 

dimensionality and "layering"

1. Dimensionality is the number of neurons in the layer(s).

2. "Layering" is the number of layers.

3 From this follows the next question, change of neurons in layers, from layer to layer?

4. Number of learning cycles from the number of layers, dimensionality and number of patterns (paternals) - (optimally)?
 

What is "change of neurons in layers", is it a process of modification of synaptic weights of neuron(s) during training of NS, or search of optimal architecture of NS by gradual change of number of neurons in layers during optimization?

Well, the number of training cycles is defined by achievement of a minimum of generalization error and is not directly related to the number of layers (and other things), although it weakly non-linearly depends on number of neurons and number of layers. It depends on the "ruggedness" of the multidimensional feature surface that the NS builds to find its global minimum. If the network works correctly, then 50-100 epochs of training by the method of Backward Error Propagation is enough. However, it will take a lot of effort to get it right.

1...34567891011121314
New comment
Reason: