Machine learning in trading: theory, models, practice and algo-trading - page 1227

New comment
 
toxic:

There are a lot of chips in general it's hard to say anything useful, and specifics, well, you know...

There are a number of "rules" that allow to avoid gross mistakes when building features from time series, in particular one of the most violated is banal "mixing" of features with targets, features should be strictly from past, and targets from points strictly from future, such separation should be at the level of algorithm of series transformation into dataset, and not as everybody does all kinds of indicators, then they cut into train and test, shift something somewhere and so on. You have to cut the initial series and then use sliding windows (past and future) to run through the series and get features and targetets separately for Lerne and test, or even for validation. Of course, you can do it with indicators, if you know for sure that the indicator for features is not looking forward, and for the targets it is not looking backward. There is a number of even more subtle errors, but I won't mention them now.

The transformations themselves may vary from the trivial (returnee, variation, volume variation, delta of the stack, distribution of deals, etc.) to all sorts of exotic ones.) up to various exotics, gradients of horizontal levels of traded volumes, "patterns" (what the hell without them))), and dozens of specific custom statistics obtained by "inspiration" or clustering that turned out to be useful, such as "trend/flight" mentioned by Mr. Innocent above, as well as "order/haos" and the like. Some statistics are valid for different time scales, some are not, some features work for some instruments, others do not, you need to be able to filter and select the features for the target. There are a lot of things, standard ARMA models , GARCH ... Mid and long term macro prediction as features, etc. I haven't got around to doing NLP\NLU for analysis of text streams, from social networks, etc. That's exactly where dip-lernig will be needed.

When I was repeating articles of Vladimir Perervenko I made an additional experiment - without additional indicators(digital filters for last articles), i.e. only on prices. The result was only a couple of percent worse. And even then I think it's just less successful training of NS (different mixing and initialization of weights and OHLC were not from the article, but from my server). NS can easily make any indicator inside itself if it needs it for predictions. I think there is no need to guess which DF with what parameters (LPF/FTF/bandpass or some MA) is useful for prediction. The NS will do everything from the OHLC.

But about more subtle errors it is still interesting to know...

It's a pity that useful information on MO for BP is scattered over 1200+ pages. If not working ideas, then at least dead-end ones like ZZ, PCA and forward/backward peeks.

 
You arenot the only ones who know the difference between a trend and a flop:

So the "grail" can only be in the context of ultra-hft, or rather was there when there was no such murky regulation, or an insider, which is available to guys who without it can just print dough in a crisis and buy their bonds with it))

Here we go again...

If you do not get tired of scaring people with your trillions and insides, you know the saying "the mutt barks and the caravan tracks", we do not need your insults to the brokerage company where you left your last hundred bucks, the world is not fair, you were unlucky, and I am lucky, I will make a hundred bucks, and if not I will try again, until I get it.


Cry to your girlfriend or your mother, don't stop us from dreaming of yachts and islands and making your dreams a reality step by step. They don't even know where to download Voronov and how to distinguish a trend from a flat.

 
Kesha Rutov:

Here we go again...

You are not tired of scaring people with your trillions and insides, you know the proverb "the mutt barks and the caravan goes", we do not need your insults to the DC where you left your last hundred bucks, the world is not fair, you are unlucky, and I will be lucky, I will make a hundred, and if not, I will try again, until I get it.


Don't stop us from dreaming of yachts and islands and making our dreams a reality step by step. There are a lot of "gurus" who do not even know where to download Voronov and how to distinguish between a trend and a flat, get lost, go back to hell.

Tell me right away - what DC do you work for? Dream propagandist.
 

I know that no one has tried, but maybe:

sorting of trades by profit/loss, accordingly at division into classes to give the most profitable trades more probability, the most unprofitable ones around 0.5. I.e. simply sort trades by efficiency and assign them probabilities, it is clear that the most effective trades will be less, and noise ones more.

Is it worth experimenting? The error should decrease?

 
Maxim Dmitrievsky:

I know that no one has tried, but maybe:

sorting of trades by profit/loss, accordingly at division into classes to give more probability to the most profitable trades, to the most unprofitable ones in area 0.5. I.e. simply sort trades by efficiency and assign them probabilities, it is clear that the most effective trades will be less, and the noise ones will be more.

Is it worth experimenting? The error should decrease?

This is regression, but not through the height of bars, but by results of deals (probably you need it for self-learning systems). I won't give any advices, because I have not been dealing with IR for 5 months and now I'm thinking which idea to try. I want to automatically decrease influence of noise predictors. After all, in the beginning of the branch it was done.
 

1 2 Partly yes, a fashionable phenomenon, but this is my opinion, "deep learning" itself is not actually regression/classification itself, but a way to extract rather simple hierarchically organized featuresfrom data of a certain type, for example pixel pictures drawn by reflected light coming to a camera from "real world", by themselves as pixel brightnesses are very bad features if you just shove them as is in a classifier, and CNN does a kind of "decorrelation of neighboring pixels" and compression of dimensionality, in several passes, similar can be done if you cluster images beaten into small fragments, in general "deep learning" is a kind of clustering in many passes and then feeding "high level features" into a regular classifier. The topic is certainly very interesting, but it's time to develop it theoretically, not just "ananalyze" with neural networks of different architecture trained by terribly slow backprop. But I repeat: so far I was not able to make diplerning friendly with market, though it is an open question, because it is too expensive to experiment in this direction, as you have correctly noticed.

3 It is not necessary to put constants in the tree leaves, you can put linear models, or more complex ones, then there will be "extrapolation" beyond the point cloud))

4 You got something wrong and forest and boosting can do regression, no problem.

3. i.e. you can combine and put other models in the leaves of the solver tree models, i wonder how it works, please give me a real example.

4. I'm not confused, and I see the problem, because so far I haven't met a regression problem with a lot of outputs (dependent variables) on the scaffold, as on neural networks.
Take, for example, the function for generating a random forest from MQL library alglib.mqh

//+------------------------------------------------------------------+
//| This subroutine builds random decision forest.                   |
//| INPUT PARAMETERS:                                                |
//|     XY          -   training set                                 |
//|     NPoints     -   training set size, NPoints>=1                |
//|     NVars       -   number of independent variables, NVars>=1    |
//|     NClasses    -   task type:                                   |
//|                     * NClasses=1 - regression task with one      |
//|                                    dependent variable            |
//|                     * NClasses>1 - classification task with      |
//|                                    NClasses classes.             |

i.e. when the number of dependent variables is more than one, this function can only solve the problem of classification. Bring your implementation with a solution to this problem, and please in the code, after all we are on a forum of programmers :)

 
Maxim Dmitrievsky:

I know no one has tried it, but maybe:

sorting of trades by profit/loss, accordingly at division into classes to give the most profitable trades more probability, the most unprofitable ones around 0.5. I.e. simply sort trades by efficiency and assign them probabilities, it is clear that the most effective trades will be less, and noise ones more.

Is it worth experimenting? The error should decrease?

If it is in the tester and in order to get to the correct entry points indirectly, through the profitability of trades, then why not find them immediately and ideally, by the price series, though I already asked this in the Monte Carlo branch:)
 
Ivan Negreshniy:
If it's in the tester and in order to find the right entry points indirectly, through the profitability of trades, then why not find them at once and ideal, by price series, though I already asked it in a Monte Carlo branch:)

then there will be few examples and on the new data ns will become a blind lost, it is necessary that she "saw" in her life as much as possible

you can not just take an ideal zigzag - you will not be able to find features then, it should be a double-edged process of trying everything simultaneously :)

I'm already thinking about whether to take more classes instead of 2

 
Maxim Dmitrievsky:

then there will be few examples, and on the new data ns will become a blind lost, it is necessary that she "saw" in her life as much as possible

It is logical, but on the other hand it may be better for her to see less and trade only the sure thing, than to constantly signal and raise the risk.
 
Ivan Negreshniy:
is logical, but on the other hand it may be better to see less and trade only a sure thing, than to constantly signal and increase the risk.

I always use a test sample to detect errors, this "sure thing" only appears to the human, while we "think" in a different way.

I think the way a neural network thinks... it doesn't care if it's a sure thing or not, as long as it can generalize

if you put any crap in the 0.5 range, and the emissions are quality trades, would the generalizing ability improve... you should check, of course, because you can ' t do it by eye
1...122012211222122312241225122612271228122912301231123212331234...3494
New comment
Reason: