Machine learning in trading: theory, models, practice and algo-trading - page 262

 
The process:

It is necessary to determine the channels, everyone will form the signs himself, I suggest a second "slices" for our futures: flippers per second bid, offer, average tick value per second, delta in the pile, the volume of purchases and sales, separately and change of open interest, for forex currency pairs flippers bid, offer, average tick, for foreign indices price and change per day example in the attachment.


The first thing to do is to deal with one or two rows, a couple of features and one aspect ratio on a small series (1000-10000 samples), and then start with 100500 features and aspect ratios.

For example let's take euro and yen to the buck on minute with MT, then add 2 - 3 TRIXes (RSI, Stochastic, etc.) as features and ZZ as a target and understand in details how it ( won 't) work and why and add series, features and targets as needed, enjoying the growth of model performance until it comes to some limit. It's always easier to scale up when there's transparent prototype, but it can't be managed with a bunch of rows at once, there are many degrees of freedom, as a rule, it ends up with thoughtlessly configuring a bunch of parameters without clear understanding of the essence of the process.

 
This is not the case with indicators thatdo not have any prediction properties at all:

First, you should deal with one or two rows, a couple of features and one tag, on a small row (1000-10000 samples), and then create 100500 features and tags.

Since they've started to share their wisdom, I'd say that the first thing you need to ask yourself is the following questions

1) what drives the market in general

2) How can you predict it?

3) How to fight with non-stationarity.

And to pile up all the indicators that do not work, and the MOE itself will not understand it, believe me in my experience, not even ..... (Moreover, I have very working "features", but I can't teach the MO to understand those "features" yet) What to say about indicators without any predictive properties?

 
mytarmailS:

Since we have started to share wisdom, I would say that first of all we should ask ourselves the following questions

1) what drives the market in general

2) how this can be predicted

3) How to fight non-stationarity

And to pile up all the indicators that do not work, and the MOE itself will not understand it, believe me in my experience, not even ..... (Moreover, I have very working "features", but I can't teach the MO to understand those "features" yet) What to say about indicators without any predictive properties?

"What drives the market" - these are models of the subject area.

"Non-stationarity" are time series models.

These are two non-overlapping approaches.

 
Dmitry:

These are two non-overlapping approaches.

And I don't cross them, but to predict the market you have to answer these questions, if you are not an insider
 
mytarmailS:
And I don't cross them, but to predict the market you have to answer these questions, unless you are an insider

If you use time series models - why do you need to "what moves the market"?

For time series models, all the information you need is in the price

 
Dmitry:

If you use time series models, why do you need to know "what drives the market"?

For time series models, all the information you need is in the price

Answer why MO trained on markettime series doesn't behave adequately on new data?

Looking for an answer to this question we will have to deal with the question "what drives the market" and solve the problem of non-stationarity, in short all the things I mentioned above, is nothing new

 
mytarmailS:

Answer why MO trained on the market time series does not behave adequately on new data?

In search of an answer to this question we will have to deal with the question "what drives the market" and also address the issues of non-stationarity, in short all that I have voiced above, nothing new

Because the data is non-stationary.

What's the point of training a model on a chunk of series if the time characteristics of series are completely different on another chunk?

 
Dmitry:

Because the data are non-stationary.

What is the point of training a model on a piece of series, if the time characteristics of the series are completely different on another piece?

Well, it's true, but that's only half the problem, it's purely a question of non-stationarity.

But there is another question - if the market data to make statsionarnymi then it turns out that anyway they are not predictable at least by leaps and bounds, here comes the second question "what makes the market".

 
mytarmailS:

Well, that's true, that's true, but that's only half the problem, it's purely a question of non-stationarity.

But there is another question - if the market data to make statsionarnymi, it turns out that anyway they are not predictable at least at a glance, that's where the second question "what drives the market" comes up

And how is the question of non-stationarity solved?
 
Dimitri:
And how is the issue of non-stationarity resolved?

I personally solved through dtw, now I found an interesting thing through spectral analysis in particular "SSA" although with skill I think the Fourier or "PCA" will do as well

You see, I'm not trying to make the price statsionarnaya, I just use those methods that are "immune" from non-statsionarnosti

Reason: