Taking Neural Networks to the next level - page 33
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
3. the post-processing
Let's talk about the scale of our desired labels compared to the outputs of the neurons in the last layer, i.e. the output layer: Considering a standard neuron, the calculation method for updating its value is always the same: inputs times weights --> plus bias --> activation function. Some more fancy neurons like LSTM cells have on top of that their so-called "gates" (activation gate, input gate, forget gate, output gate) with their individual weight matrices (and bias and an activation function of their own), but the end result is comparable: in the end, a cell's output is always the result of some activation function. Like for any neuron, this is also true for the neurons in the output layer, which is why the range that the output values can be within is dictated by the chosen activation function of the last layer.
This can be a problem, or at least we need to chose the model's parameters in a way to deal with it. If our "true" labels for example are in a range between 0 and 1000 and we have the sigmoid function (just as an example) as the chosen activation function of the last layer, this just doesn't match, because the sigmoid function returns only values between 0 an 1 (or more precisely: sigmoid(x) is between 0.5-1 if x is positive, or 0-0.5 if x is negative). "Doesn't match" in this case means the the network will almost always put out a result of +1 (because more is not possible with sigmoid) and we almost always will end up with a gigantic error between output and label and the network can't do anything against it.
If you take another example, the ReLU function, results can range between 0 and positive infinity. If the labels are very small values, this can also be a not so perfect match, although it could be done.
To summarize: we either need to chose an activation function (for the last layer) whose range of possible results matches with the labels, or we need to rescale the labels before we compute the errors into the backpropagation process. How we scale the labels then essentially depends on the activation function. If we have tanh as last activation function (for example), which can put out values between -1 and +1 one, then a simple min-max scaling method that squashes the labels between -1 and +1 might be the obvious idea, whereas for example normalisation (1=one standard deviation, zero mean) would be a bad idea, because the label range would exceed the output value range.
If we have scaled the labels to fit for backpropagation, then of course we have to do the opposite with the results of forward propagation in order to come up with usable results on the same scale as the labels.
If the labels consisted of a data series that we made stationary, like in our autoencoder example, we need to reverse this, too, of course.
Programming a Deep Neural Network from Scratch using MQL Language
I enjoy reading your thread a lot to get into the topic. Although I have to ad that cats, dogs, speech recognition, the prediction of gender, heart disease or diabetes are very stationary, seasonal examples with lots of inputs that are often very consistent with the results they produce. Stock market on the contrary is very non-stationary, only partially seasonal with inputs which rarely give a clear indication of how a system will behave even in the very near future.
You are partly correct, knowledge here presented is basic level. You can get that much understanding from any 3h video tutorial on the web. Any successful trades/short histories that may be presented are mostly conducted in favorable conditions, that is stable are rarely reversing trends.
What next level AI tends to aim for is greater level of abstraction, achieved by skillful insertion of "non linear" operators. These MAY lead to better performing algorithms in any conditions.
I have a question on this subject. I hope to write as clearly as possible.
I have been trading with neural networks (LSTM, to be precise) and when I trained them, the data was normalized.
But if the data is normalized, should I normalize it every time I perform live operations? I currently do it, but now I have my doubts about whether it is a good idea or not.
I mean ... if the data is normalized every time a new candle is formed, every time the values would be different because I am adding a new candle, and if it is normalized, then the candle that had a value of 1 (for example ) could not have it anymore. Isn't there an inconsistency?
According to art yes, you should input the data in the AI the same way as you did while training it. My own experiments with LSTMs showed that they will give a reasonably good looking answer even if you provide it with not processed data.
That itself should lead you to a question "if i have a system that works so haphazardly that doesn't obviously change behavior on "bad" inputs, is it any good"? Realizing just how chaotic markets are, any AI system will give you as much rope to hang yourself as a coin flip.
Experiments with neural networks (Part 1): Revisiting geometry
is there profitable NN EA? if so its the end of markets and so unlikely to exist
No Recommendation is allowed for EAs. Keep searching, I mean change your search keywords.