Machine learning in trading: theory, models, practice and algo-trading - page 40
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
The dimensionality of array X
In your case it would be
Naturally, if the number of examples is 7000 and the conversion length = 8.
This "rnn" package is more of an exercise in programming than a product to use. See RSNNS instead.
There are several recursive networks. Well, in more depth, "PyBrain", "keras" in Ruthon
Good luck
I have a strange problem with cross-validation of the trained model. More precisely, the problems were always there, I just did not notice.
My actions are about this: There are different indicators for eurusd for a year. I divide data randomly by rows in ratio of 70%/15%/15% for training/validation/validation.
Further, this data contains 9000 predictors. I genetically search through their variants, train a PCA model based on each subset of predictors. I train the model on a training sample, determine error. I test the same model on both validation samples, determine error on them.
The final result of a fitness function is defined as follows:
max(error_train,error_validate1,error_validate2) - min(error_train,error_validate1,error_validate2)
The point is that the difference between minimum and maximum errors should be as small as possible (the so-called "Fomenko criterion" :) ).
All seems well, the model achieves an error of about 25% on all three samples. But, on the fronttest, the error suddenly rises to 40%. I wondered a bit, changed random.seed for generating samples of strings (not predictors, just strings), and for the previously defined set of predictors (the best one) I already got training/validation/validation error of 25%/35%/35% instead of 25%/25%/25%, and again the same 40% for fronttest. The error on validation samples increased by +10%, very bad.
So it turns out that genetics finds the best version of predictors for a particular training, and particular validation samples. If you scatter the rows over the samples in a different order, the trained predictive model no longer works adequately with the same predictors.
Are there any standard ways to deal with this so that the trained model gives approximately the same errors for any distribution of rows over the samples? This is a serious problem. I can, for example, generate training and validation samples all over again every generation of genetics. Or make two different sets of row samples, train two PCA models on them, and take the worst result for the fitness function. Does this make sense?
I have a strange problem with cross-validation of the trained model. More precisely, the problems were always there, I just did not notice.
(so-called "Fomenko criterion" :) ).
I liked the criterion very much. Thank you.
But it is not completely stated by you.
My criterion MUST include the fronttest. If you look at all my posts, I always wrote exactly about fronttest, under which, according to my ideas, should be conducted on a piece of observations, which by time (by index) are separated from the first three sets.
This is something I ran into back in rattle, which divides the provided set the way you did. You get good data, I got less than 10% on all three samples, and then you take a new file and your 40% error is still a good result! I've had worse.
So my criterion for overtraining is:
1. we divide the original set into two parts, WITHOUT any random focus, purely mechanically, in ascending order of index
2. We divide the first part by any of the random methods into three parts: learning testing and validation.
3. if on these three parts error is approximately equal, then go to p.4
4. Run the trained model on the second part. The error must again be approximately equal. This is fundamental, since when trading we always trade outside the training sample and get sequentially (not randomly) bar after bar.
If we get approximately the same error on all 4 points, the model is not retrained.
If there are considerable discrepancies, the model is retrained. I haven't found any tools in R to get rid of the noise. I have my own homebrew one, here is another article about PCA.
Anton Zverev
Let's not talk that way, people who learn and share their experiences here are willing to help each other, while you take the position of saying you are stupid here and I know everything) You'd better help me understand what you think and experience is right.
I agree with you that just giving BP is not enough, you need to compress the information and discard unnecessary things that prevent making the right decision, ideally to 0 or 1 those buy/sell, that isIf we have 10 indicators (that I do not believe in) and we have cut off 9 of them, for example RSI, we will still have too little, because Indus has its range and it turns out it does not work with values from -70 to 70, so it needs to compress and so on.. the question is how to do it?
I have thoughts about it, but I don't have enough knowledge to implement such a selector, yet...
My first try was a long time ago, I looped back from the current price and searched for a nearly identical situation in the past, then these situations were sorted by the result, how they ended up, for example, I had a current situation and found 10 analogues for it in the past 8 analogues ended with a price increase, 2 ended with a fall, so it will grow ... But the horror )) is that it was vice versa, the price fell often and strongly in these situations, with a strong bias towards buy, and then it often retested tick by tick...
Then I made a kind of indicator, took the cumulative sum of all buy prices and also the sum for profit, made their difference and got a certain index, when I compared it to the price it turned out to be opposite to the price, the correlation was -0.7 to -0.9, so simply said, the market goes against its own statistics, this is something to think about and reconsider
toDr.Trader
Maybe this will help you to understand a bit.....
I found a faster and easier way to see this affect.
4. Run the trained model on the second part. The error should again be approximately equal. This is fundamental, since when trading we always trade outside the training sample and get consistently (not randomly) bar after bar.
I see my mistake, thank you. It turns out that you have not 2 but 3 validation samples - one of them is taken strictly from the last records of the source file, without randomness. The other samples are random. I'll give it a try.
I once tried to take the first part of training file for validation, also strictly by indexes, also without random. The idea was that if trained model shows small error on data before training period, it will show the same error on data after training period. It turned out not so, errors before the training period and after the training period in the end I did not correlate at all.
Then I implemented such an indicator, took the cumulative sum of all buy and sell forecasts, plotted them all and got some index, and when I compared it to the price it turned out to be opposite to the price, the correlation was -0.7 to -0.9, simply speaking the market goes against its own statistics,this is something to think about and reconsider
Does this method always work for you, at any time interval? Of course it looks strange, but the main thing is that it works. I had similar observations with my untrained model. Sometimes it gave predicted results that were the exact opposite of the real ones. But sometimes it worked correctly. During a very long test, while training the model and checking it on different time intervals I understood that on average it gave the right answer 50% of the time, that is, it was useless, you could flip a coin to make a prediction and get the same result. After that I made a decision, that the model should give right result without any inversion, filtering, collective solutions of many other models. Otherwise you can get yourself into a trap, where chaos and randomness rule.
Does this method always work for you, at any time interval? It turns out strange of course, but the main thing is that it works. I had similar observations with my untrained model. Sometimes it gave the predicted results strictly opposite to the real ones. But sometimes it worked correctly. During a very long test, while training the model and checking it on different time intervals I understood that on average it gave the right answer 50% of the time, that is, it was useless, you could flip a coin to make a prediction and get the same result. After that I made a decision, that the model should give right result without any inversion, filtering, collective solutions of many other models. Otherwise you can get caught in a trap, where chaos and randomness rule.
Yes always, and at any stretch of time (but there are nuances with sampling), the output is almost an exact copy of the price, but inversely correlated, my point is. This way you can see how a neural network is trained and what to expect from it, you can make several conclusions
1) this is the most typical situation - if a network is inversely correlated, that means it is trained, otherwise it wouldn't be inverse correlated, right?
2) It is just random, so a network has not learned anything.
3) A net works together with the market (I managed to achieve this effect only on a non-standard target) - this means that everything is OK
4) And the main conclusion (this is important for you D.Trader) if you build your neural network forecasts and it will go in the opposite direction as in conclusion 1, no optimization, genetics, crossvalidation and other charms will help you absolutely not to think about it, it's logical and obvious
1) this is the most typical situation - if the network is inversely correlated, it means that it is trained, otherwise there would be no inverse correlation, right?
4) And the most important conclusion (it's important for you D.Trader) if you make your neural network forecasts and it will go in the opposite direction as in conclusion 1, no optimization, genetics, crossvalidation and other charms will help you in ANY way, think about it, it's logical and obvious
1) stable inverse correlation at different training intervals is a bit strange, but the network has learned something, I agree that the result is good.
4) I disagree with this one. If you use binary classification (target variable is "0" or "1"), then fitness function of genetics or optimizer can be easily changed to achieve "reverse" result, just subtract current result from 1.
For example, genetics picks up a certain set of predictors. I train neuronics on them, do crossvalidation, find average error. The error is let's say 0.4. The "inverse" result of the fitness function can be defined as 1-0.4 = 0.6. Genetics will try to decrease error from 0.6 to zero. It will result in the set of predictors with fitness function value 0. But it means that the real error of the neural network was 0+1 = 1. That is, the resulting neural network is wrong in 100% of cases, and this is the inverse correlation.
In general, you can use the same genetic algorithm and crossvalidation to train neuronics that will give the result opposite to the desired one, it will turn out as you did, but on purpose. But it is not clear why to do it? :)
I would also add that in binary classification with target variables "0" and "1", the worst error is 0.5.
Error == 0 is good.
Error == 1 is your inverse correlation neuron, you can just flip the result and it will be ok. But it's better to find out why, it's a very atypical situation.
Error 0.5 means that the result is completely random and the model is useless at all.