Market etiquette or good manners in a minefield - page 21

 
grasn писал(а) >>

While I'm creating, I have to look like an artist! But when I'm coding ... :о)))

PS: Serega, it's almost monopenic in MathCAD, but in C++/FORTRAN ... will be different, but in MathCAD - to put it mildly not (for this particular case). And if you use mean, it'll count even faster, and if you use corr, for example, your "fast" algorithm will turtle :o). How they do it, I don't know, but as for "summing" operator - it's much faster than loop. If you don't have it that way - then put a fresh version.

Got it.

Thanks for the info!

 
Neutron >> :

Don't forget that it's advisable to teach all this stuff at every countdown - it won't be easy.

Now I get it! I was counting all the committees in one pile...

I'm not going to increase the number of neurons in the hidden layer. I don't see why, it works fine with two.

If you don't mind, I would like to go straight to ORO, as I already have a two-layer mesh ready and working (even with scales "from the background"), if interested - I can lay out the code.

 

So far, here's what I've done:



d = 17;

w = 17*3 + 4 = 55;

P = 4*55*55/17 = 712;

------------------------------------------------------------------------


The ORO algorithm so far is figured out like this (only for output layer weights):

// ----------------------------------------- ЭПОХА ---------------------------------------------------

for(int i = cikl; i > 2; i--)
{
out = OUT(i); // Получаем выход сетки
target = (Close[i]-Close[i+1])/Close[i+1];--------------------// Получаем целевое значение
delta = target - out;-----------------------------------------// Вычитаем выход сетки из целевого значения
pr = th(out);-------------------------------------------------// Вычисляем производную выходного сигнала
pr = 1 - pr*pr;-----------------------------------------------//

for(int n = 0; n < 4; n++)
{
Corr[n] += delta * pr * D2[n];--------------------------------// Вычисляем ошибку для каждого веса и суммируем её за всю эпоху
Norm[n] += Corr[n]*Corr[n];-----------------------------------// Считаем норму вектора коррекций для каждого веса за всю эпоху
}
}
// ------------------------------------- КОНЕЦ ЭПОХИ --------------------------------------------------

// ----------------------------------- КОРРЕКЦИЯ ВЕСОВ ------------------------------------------------

for(i = 0; i < 4; i++)

W2[i] += Corr[i]/MathSqrt(Norm[i]);



It seems to work! But something is still wrong. Over 1000 epochs all the weights have become very small:

W2 [0] = 0.0876 W2 [1] = 0.0772 W2 [2] = -0.0424 W2 [3] = -0.05

What did I miss and how do I now push the error to the input weights?

 
Prival писал(а) >>

Look at the private messages. I apologise for writing here, but they often don't reflect the messages.

Sergey, check your private messages.

to paralocus

I will reply to you a bit later.

I'm working on the input data for the NS and the idea came to my mind. By the way, why did you multiply the number of weights by 3 when calculating the training vector length? I understand you're supposed to have w=17+4 and that's it! You're training each committee member with their own personal vector...

 
Neutron >> :

...By the way, why did you multiply the number of weights by 3 when calculating the training vector length? I take it you should have w=17+4 and that's it! You're training each committee member with his/her own personal vector...

In this case the input vector is the same. It is read once at each count and then each committee member "looks at" it through their personal "points" (weights) and makes a judgement.

Of course, it can be read for every member of the committee separately, but what's the point if the result will be the same? The input vector is 16 consecutive increments of the quotient + a single input.

А... I get it! Yes, I used to have two different input vectors: the kotir increments with dt according to Fibo series - 2,3,5,8... and the other by Luke's series - 3,4,7,11... and there were only 2 committee members. Back then, each committee member read a different vector, but now I've simplified things to focus on understanding the ORO. So, the vector, in this case, is one.

Could the weights have dropped so much because I only taught the second layer?

 

Well, it still turns out: w=17+4 and each member has the same vectors.

Paralocus, here's what I think. Let's not get ahead of the curve and solve a bilayer nonlinear Perspectron with nonlinearity output in MQL and get a positive result. Then you'll move on to a committee of networks (if needed).

What do you think?

 
Neutron >> :

Well, it still turns out: w=17+4 and each member has the same vectors.

Paralocus, here's what I think. Let's not get ahead of the curve and solve a bilayer nonlinear Perspectron with nonlinearity output in MQL and get a positive result. Then you'll move on to a committee of networks (if needed).

What do you say?

OK! So we take all the commissars and the chairman out. Sort of "gangbusters bullets"... -:) >> let's do it.

P/S except one - the first one

 

Well, let's go then!

1. find the error at the output of the last neuron: d2out=X[-1]-OUT2, (I mean numbering as MQL and this is the "future" count).

2. Recalculate it taking into account the non-linearity at its input: d2in=d2out*(1-OUT2^2)

3. The same error goes to the output of each neuron of the first layer by its axon! Now, this error at its outputs is d1out and we repeat procedure (2) recalculating at its input: d1[1]in=d1out*(1-OUT1[1]^2) and in the same way for the second neuron of the hidden layer.

 
Are the weights corrected by calculated error, or not yet?
 

No.

First, we calculate the micro-correction for each weight: dw=in*din - delta rule - we multiply the input signal by the error applied to a particular input. Then we summarize it in a separate summator for each weight over all samples of the training vector. The weights are not corrected within an iteration of the training epoch. Only microcorrections and their squares are accumulated (in another adder).

P.S. I corrected the error in the post above. The derivative is found as follows: *(1-OUT2^2).

Reason: