Machine learning in trading: theory, models, practice and algo-trading - page 74
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Yury, I decided to use the holy of holies and make a grail, but I got an inscription that predicted values cannot be more than 10. Is this a deliberate restriction or a limit of the algorithm???? Because more than 10, it is extremely relevant so....
After 10 there may be an overflow of int type, and it has only 32 bits, i.e. before the overflow it holds numbers not exceeding the value of 2^32=2147483648. Just a little over two billion. You can of course make counters with larger bit depth, but calculation speed will be several times slower.
That's why Dr.Trader couldn't run full libVMR rewritten in R - too many calculations and it eats a lot of memory.
After 10 there may be an overflow of int type, and it has only 32 bits, i.e. it fits numbers without an overflow not exceeding 2^32=2147483648. Just a little over two billion. You can of course make counters with larger bit depth, but calculation speed will be several times slower.
After 10 there may be an overflow of int type, and it has only 32 bits, i.e. before the overflow it holds numbers not exceeding the value of 2^32=2147483648. Just a little over two billion. Of course, you can write counters with higher digit capacity, but the speed of calculations will fall by several times.
That's why Dr.Trader couldn't run full libVMR rewritten on R - too many calculations and it eats memory.
I'll tell you one story. I remember, I asked you about paralleling calculations. So that I could count more inputs on the math coprocessor. One asshole, who rewrote LibVMRx in C++ and sort of paralleled it, he had a powerful computer and 8 columns was counting in 40 minutes, as he said. In the end he did not send me any info or programs and was giving me excuses and bullshit. Although I have given him all the data and was ready to share the system and considerations for building the model. But he just cut me off. There are people like that. Blow in and out ....
I paralleled everything in the latest version 6.01. Nothing else works. If it were possible, I would parallelize more. I was doing it for myself, i.e. I need it all to be computed faster.
Now I've come up with an algorithm to calculate weak predictors so that they can be removed. It will take a long time to calculate, but it will be worth it, because instead of a weak predictor, you can substitute stronger ones. Or, you can get rid of the weak predictors, increasing the speed of calculations and the generalizing ability. Because for trading, you have to constantly run a recalculation of the models as soon as the market starts to go cheating. And that's the way the market is, as soon as other traders get merged, they change their strategies and you have to adjust to them all over again.
I paralleled everything in the latest version 6.01. Nothing else works. If it were possible, I would parallelize more. I did it for myself, i.e. I need to calculate it all faster myself.
Now I have come up with an algorithm, how to calculate weak predictors, so that they can be removed. It will take a long time to calculate, but it will be worth it, because instead of a weak predictor, you can substitute stronger ones. Or, you can get rid of the weak predictors, increasing the speed of calculations and the generalizing ability. Because for trading, you have to constantly run a recalculation of the models as soon as the market starts to go cheating. This is how the market is, as soon as other traders merge, they change their strategies, and it is necessary to adapt to them anew.
That's right. I look forward to the changes.
You can manually go through the predictors one by one, but it's long and painful, although it's also the right direction. We need to make it long, but it should be automatic and in a reasonable amount of time, and not so painful.
Mihail Marchukajtes:
We are on the same side of the fence. I also believe that the market is changing all the time and it needs to be adjusted each time something happens or news is released. Do not look for a grail in 5 years, as some people do here. This is utopia, but I'm already exhausted from explaining it...It's possible to find weak long-playing grails. I had a few of them. But my profit is on the verge of falsification, my profit is penny-ante even if I have a good depo.
It is theoretically possible to find a strong long-playing grail, but the probability is probably zero point zero?
The surest way:
HM... Look now trained model, both models got the same values of generalization level, i.e. the market segment is sawn the same way by both models. I think this is the key to choosing the appropriate model. I continue to train, look for inputs, etc.
Look in the java file. If there are lines "Variable got under reduction", they indicate weak predictors. Such predictors are excluded from the model code - they are not calculated anywhere in the code of the binary classifier.
However, if all predictors are approximately the same, such lines do not appear.
Here is an example (in bold). The sixth predictor v5 (counted from zero) - the seventh column in the CSV can be replaced with any other, stronger one:
Look in the java file. If there are lines "Variable got under reduction", they indicate weak predictors. Such predictors are excluded from the model code - they are not calculated anywhere in the code of the binary classifier.
However, if all predictors are approximately the same, such lines do not appear.
Here is an example (in bold). The sixth predictor v5 (counted from zero) - the seventh column in the CSV can be replaced by some other, stronger one:
double getBinaryClassificator1(double v0, double v1, double v2, double v3, double v4, double v5) {
double x0 = 2.0 * (v0 + 1.0) / 2.0 - 1.0;
double x1 = 2.0 * (v1 + 1.0) / 2.0 - 1.0;
double x2 = 2.0 * (v2 + 1.0) / 2.0 - 1.0;
double x3 = 2.0 * (v3 + 1.0) / 2.0 - 1.0;
double x4 = 2.0 * (v4 + 1.0) / 2.0 - 1.0;
//Variable v5 got under reduction
double decision = -0.23641879194630872 -0.10890380313199105 * x2
-0.14546040268456376 * x0 * x2
+ 0.07270693512304251 * x1 * x2
-0.07182997762863534 * x3
-0.07383982102908278 * x0 * x1 * x2 * x3
-0.4362541387024608 * x4
;
return decision;
}