Machine learning in trading: theory, models, practice and algo-trading - page 190
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
That is, it is useful to do this. So instead of 4 partitions, which is obviously not enough, you should do 40 partitions. For 4 cores will take 10 times longer to compute, but I suppose time can be sacrificed in favor of robustness.
Not obviously.
For example, waiting 10 hours for calculations instead of 1 hour is unacceptable for daytrading. Even if we leave the computer overnight, we will obtain a model based on inevitably outdated data.
So there needs to be a sensible compromise between computing time and model quality. And the best way is to parallelize everything that can be calculated in parallel, while everything that cannot be parallelized should be calculated sequentially.
As a last resort, you can upgrade your computer to a larger number of cores, or build a computing cluster of several personal computers.
I am not even talking about the fact that machine learning algorithm code often has potential opportunities for further optimization.
It is also possible that some part of multitasking can be transferred from CPU to GPU.
I.e. there are a lot of potential solutions to the problem (the list can go on and on) and "making a hump" in software is not the best of them, and as experience shows, it is often the most inadequate.
It's not obvious.
For example, it is unacceptable to wait 10 hours for calculations instead of 1 hour for the day-trading. Even if you leave the computer overnight, you will get a model on obviously outdated data.
So a sensible compromise between computation time and simulation quality is necessary. And the best option: all that can be computed in parallel should be paralleled, what cannot - compute sequentially.
As a last resort, you can upgrade your computer to a larger number of cores, or build a computing cluster of several personal computers.
I am not even talking about the fact that machine learning algorithm code often has the potential to further optimize.
It is also possible that some of the multitasking can be moved from CPU to GPU.
I.e. there are a lot of potential solutions (the list can go on and on) and "making a mess" in software is not the best of them and, as experience shows, is often the most inadequate.
I do not insist on the "hump option", just asking: the more variants the data will be divided into, the better training can be obtained by doing an analysis of the results. Let's say that in 90% of cases we see that the model shows adequate results on the test data, and only in 10% we get overtraining, which means that the model itself is worth something. And if it's the other way around, it's worth recycling. And if we divide the data into only 4 different variants, the probability that we will get an overtrained model is extremely high.
Again, I'm not touching on the "hardware" aspects, just clarifying the "software" ones.
...
Not having advantages over the known..... but nobody will believe)))
Give a concrete example of known ... that "have advantages".
And from you nothing but unsubstantiated criticism, which always ends with the fact that the sample does not give, do not show the software (all strictly classified, witnesses removed). But you draw some unrealistic figures that no one but you can neither confirm nor deny.
Banal question for you: if, according to quotations, you "have a possibility to get" 92% with kopecks of generalizing ability, then why are you still engaged in empty criticism of some not "having advantages with known ...", but not engaged in buying: factories, newspapers, steamships, islands, yachts, etc.? And when will I be able to admire your face on the cover of Forbes?
There are clear rules for treating interactions in linear models. They are a little more complicated than the linear combination treatment:https://www.r-bloggers.com/interpreting-interaction-coefficient-in-r-part1-lm/
But you have to dig through a lot of combinations to find meaningful interactions. Now that's a bummer.
Had a look, thanks.
My perplexity only increased from reading it, albeit on a new understanding of the word "interaction".
My perplexity is based on the fact that any number in statistics must have some kind of content.
Take this equation lm
zz ~ rsi*stoch
Individually each indicator has some meaning, but what is the meaning of their product? What is the meaning of this "interaction"? Or do we just get a new predictor of unknown content and see how it affects the target variable?
I looked it up, thank you.
My perplexity only increased from reading it, albeit on a new understanding of the word "interaction.
My perplexity is based on the fact that any figure in statistics must have some content.
Take this equation lm
zz ~ rsi*stoch
Individually each indicator has some meaning, but what is the meaning of their product? What is the meaning of this "interaction"? Or do we just get a new predictor of unknown content and see how it affects the target variable?
It's funny listening to you sometimes )
No new entity is physically obtained from the multiplication, although the interaction counts exactly as a multiplication in classical regression.
The slope angle for the first predictor will depend on the level of the second predictor. This is the essence of interaction. It's simplistic. There are nuances there. But knowing all coefficients of such a tier we can say that, for example, x1 level increase by 1 will increase the target level by 0.1 at x2 level = 3. Something like this.
I don't insist on the "humped variant", I'm just asking: the more variants the data will be divided into, the better training quality you can get by analyzing the results.
...
Again, I'm not touching on the "hardware" aspects, only specifying the "software" ones.
Let's say, any fool can take and stick a loop of 10 iterations, thereby increasing computation time by an order of magnitude for the sake of insignificant reduction of probability of irregularity of pattern distributions in parts of a sample. The source code of jPrediction is in the public domain and is available for modification to anyone who feels like it.
I'm much more interested in spending my time and effort to achieve goals in a more promising way: creating an algorithm that is less demanding on computational and computer time resources to train models with higher generalizability.
Let's say, any fool can take a cycle of 10 iterations and thus increase computation time by an order of magnitude for the sake of insignificant reduction of probability of irregularity of pattern distributions in parts of the sample. The source code of jPrediction is in the public domain and is available for modification to anyone who feels like it.
I'm much more interested in spending my time and effort to achieve a similar goal in a more promising way: creating an algorithm that is less demanding on computational and time resources of the computer.
Yury, I'm not asking you to change or modify anything in your software, and I'm not a clue about Java, so, although I wanted to see your code, I still don't understand anything there.
I'm just asking, theoretical question, what do you think will be good, if it is possible to break the data into as many variants as possible? Yes, or no.
...
Just asking, a theoretical question, do you think it would do any good to be able to break down the data into as many options as possible? Yes, or no.
There is no point in discussing tasks that are trivialized by "number, not skill." For example, if one digger can dig a trench in a day at a distance of n meters, then m diggers can dig the same trench in the same time at a distance of m * n meters. What is there to discuss? If the critical time to dig a trench, you need to divide the time remaining before the deadline in days by the distance that one digger can dig per day and provide for several additional diggers for unforeseen circumstances. The solution to the problem is trivial and there is nothing to discuss.
You can discuss tasks, which have a solution not so trivial, for example, how to increase the productivity of digging the same trench with fewer diggers for the same time? There are already emerging options, such as: replace diggers excavator or improve their shovels, etc., etc.
There is no point in discussing problems that are trivially solved by "numbers, not skill. For example, if one digger can dig a trench at a distance of n meters in a day, then m diggers can dig the same trench at a distance of m*n meters in the same time. What is there to discuss? If the critical time to dig a trench, you need to divide the time remaining to the deadline by the number of diggers and make provision for several additional diggers for unforeseen circumstances. The solution to the problem is trivial and there is nothing to discuss.
We can discuss the problems which solution is not so trivial, for example, how to increase productivity of digging the same trench with fewer diggers for the same time? There are already emerging options, such as: to replace diggers excavator or improve their shovels, etc., etc.
The answer is florid, I'll take it as a yes.
Suppose I have a rack of 64 clusters at home, each with 4 8-core processors, why would I not use it for this purpose? Especially if it will do me good.
We have been fussing for nothing in 191 pages. It's time for everyone to go home empty-handed.
See https://www.mql5.com/ru/forum/3457/page3396#comme nt_2939962
I quote:
That's it. Machine learning is no longer needed. Everyone is beaten by the regular office suite