Machine learning in trading: theory, models, practice and algo-trading - page 155

 
J.B:

This is an empirical estimate of the increase in the quality of classification with - without this factor, everything is simple, mutual information and determination in nonlinear multifactor systems do not work reliably. And figures 4-5% is certainly not a dogma, you just need to understand that using "all markets" and information flows without price dynamics of the instrument you can predict its future for a certain horizon <5% worse, that's all. That is, if you have a one minute probability of predicting the future rise in front of the asset, for example 70%, then excluding the price of the predicted series from the data for analysis you get 70 - (70-50)*0.5 = 69%, almost within the limits of noise. Well, of course if you have real-time data from all world markets and not only markets, but without inside information, and if only one instrument's price... alas, it's easier to create a terminator than to beat a market with such data, whatever AI you have made.

Well, okay.

Let's say I'm not classifying the up/down gain, I'm building a regression model. So R^2 or some other deterministic metric (e.g. robust absolute deviation metric) is fine.

Regarding mutual information, is it barefaced, or is there strong evidence that the metric works unreliably? I doubt it.

Update: I've done a lot of research on synthetic and real data using mutual information. If the dependence is stationary, the metric works well everywhere. If the dependence is on the verge of noise, the metric may show zero dependence. But in general I see no reason why it performs worse in multivariate nonlinear systems than, for example, F1. You can read it here:https://habrahabr.ru/company/aligntechnology/blog/303750/

But when I was doing classification of incremental price movement I got approximately the following picture (for 5 currency pairs together, i.e. one model for all)


That is, at least median accuracy values on 50 pending samples in the neighborhood of 57% at the maximum. For some currency pairs I reach median accuracy above 60%. This is only on time series data.

 
Alexey Burnakov:

1) This is not a naïve view. It's the direction of the search. And not necessarily a neural network. The thesis is this: you can pull information from past time series price values that is sufficient for profitable (overcoming costs) trading regardless of the time horizon of the actual forward test.

I will also post a couple of charts on this topic. I am currently preparing material for an article.

2) PS: Personally at me, taking into account the struggle with all factors leading to overtraining, and trying to take the most conservative and reliable model condition, it turns out that more than 30-40% per year (at max drawdown 25%) is not to squeeze out. But it already exceeds the median yield of hedge funds. All the other cosmic interest supposedly obtained in the long term purely on the technical analysis of the time series - this is a lie.

1) Of course you need to search everywhere, I just suggested to use more information and only, and besides an established trader cannot be led astray from his own path, you can only add information to his existing model.

2) median returnees even american hedge funds are sad below the indices, the toughest ones hardly give 15-20% on average during 10 years, although models are not allowed to trade with a Sharpe* below 2-3 and of course the capacity is $10^6-9, according to calculations everyone should have at least 20-30% but ....

 
J.B:

1) Of course, you have to look everywhere, I just suggested to use more information and only, and besides an established trader cannot be led astray from his own path, you can only add information to his existing model.

2) median returnees even american hedge funds are sad below the indices, the steepest barely 15-20% give an average of 10 years, although the models are not allowed to trade with a Sharpe* below 2-3 and of course the capacity there $10^6-9, by calculation all at least 20-30% should be but....

1) it is yes

2) well yeah... but this Sharp 2-3 how is it calculated? How do the funds calculate, or rather HOW do they determine that this is a real Sharpe estimate on a real trade?

 
Alexey Burnakov:

PS: Personally, taking into account all the factors leading to overtraining, and trying to take the most conservative and reliable model state, it turns out that more than 30-40% per year (at max.drawdown 25%) not to squeeze out. But it already exceeds the median yield of hedge funds. All other cosmic interest supposedly received in the long term purely on the technical analysis on the time series - this is a lie.

))) Hilarious!

How much leverage do you have on these 30-40% a year?

 
Dmitry:

))) Hilarious!

How much leverage do you make these 30-40% a year?

Maximum deposit load of 10% (1:10).

Well, I must say, sometimes I've seen examples of doing more for years. But it was either by hand or with drawdowns on the verge of a foul.

 
Alexey Burnakov:

Maximum deposit load of 10% (1:10).

Well, I should mention that sometimes I've seen examples of making more than that for years. But it was either by hand or with drawdowns on the brink of a foul.

I do not mean the deposit load, I mean the leverage - with what kind of leverage do you earn these 30-40% per annum?
 
mytarmailS:

Guys there is an idea, it is worth checking, I had it for a long time, I wanted to check but could not figure out the package and somehow forgot and abandoned, but here read the threadJ.B and remembered, it turns out he also did something similar:)

We are talking about cross-correlation - we can calculate how much one BP lags behind the other BP and whether there is a connection between them ...

The essence of my idea was to monitor simultaneously a large number of pairs and build something like a cross correlation matrix to compare each pair with each other and find pairs that follow each other for a while but one lags behind and trade this lag, since the market has nothing more constant than time, I think the calculations should be done constantly on each new bar, to immediately notice when a new relationship appears and also to immediately notice when this same relationship disappears ...

You can take anything, any predictor, but I think the best approach is pairs as market makers when the price of their instrument is almost always guided by one or a bunch of other instruments, and the classic indicators are unlikely to fit

The same way you can try to train a neural network on such dynamically changing predictors, in short, everything is limited only by imagination ...

I would try to implement it myself, but I'm busy with other project and don't want to spill my guts

Standard function of cross-comparison in R-ka ccf()

is an advanced package with preliminary spectral breakdown into levels and then already check for cross-correlation "wavemulcor" it also allows you to compare many BPs at the same time

Unfortunately this is a failure, pairs walk by themselves, another thing is entropy, this will be more interesting.
 
Dmitry:
I'm not talking about deposit loading, I'm talking about leverage - with what kind of leverage are you earning that 30-40% p.a.?
Ooh, I see that you do not understand what leverage is. I've indicated in parentheses 1:10 the maximum leverage (if several trades turn out to be in the market).
 
And by the way yes, I have already started to write an article, as soon as it is ready I will definitely report it in this thread. It will contain my treatise.... :-)
 
Alexey Burnakov:
I see that you do not understand what is the leverage. I've specified 1:10 leverage in brackets at most (if several deals turn out to be in the market).

10% is the deposit load.

If you have a deposit of $1,000, you load it up by 10% - you open a trade for $100.

Now, WARNING, depending on the leverage provided by your broker/coach you can buy different lots - $10,000 (1:100), $5,000 (1:50), $20,000 (1:200).

P.S. fuckerbaby........

Reason: