Machine learning in trading: theory, models, practice and algo-trading - page 288

 
Again, I would like to add and define. We predict the price of an asset. At least we try to, in any representation and processing. Whatever target we build, the point of working in the market is to predict the price. The question is what causes the price to change?????? Who could answer or at least guess? I'm interested in your opinion, then I'll give you the right answer. Well... here we go. Your guess. What is the reason for the price change?????
 
Mihail Marchukajtes:
Again, I want to add and define. We predict the price of the asset. At any rate, we try to, in any representation or processing. Whatever target we construct, the sense of working in the market is to predict the price. The question is what causes the price to change?????? Who could answer or at least guess? I'm interested in your opinion, then I'll give you the right answer. Well... here we go. Your guess. What is the reason for the price change?????

We predict the price change, by (in) how much the price moves in N seconds/minutes/hours in the future, or more conveniently the future returnee(R =(P(t)-P(t+n))/P(t)) or logreturn.

The price changes due to the pressure of supply and demand imbalance, which occurs for many reasons, from the banal exchange of large sums of currencies as a result of international trade, crediting countries, hedging currency risks, to arbitrage and speculation on insider and news, as well as information derived from market activity, weather, fashion and solar activity.

 

Here is the list of predictors and target variables that factors, shifted by the corresponding number of bars

'data.frame':   6971 obs. of  105 variables:
$ AUDUSD     : num  0.722 0.722 0.722 0.721 0.72 ...
$ CHFJPY     : num  120 120 120 119 119 ...
$ USDCAD     : num  1.39 1.39 1.39 1.39 1.39 ...
$ GBPCHF     : num  1.47 1.47 1.47 1.48 1.47 ...
$ EURGBP     : num  0.739 0.74 0.739 0.737 0.739 ...
$ USDJPY     : num  119 119 119 119 119 ...
$ GBPUSD     : num  1.48 1.48 1.48 1.48 1.48 ...
$ EURUSD     : num  1.09 1.09 1.09 1.09 1.09 ...
$ EURCHF     : num  1.09 1.09 1.09 1.09 1.09 ...
$ USDCHF     : num  0.995 0.995 0.994 0.998 0.998 ...
$ GBPJPY     : num  176 176 176 176 175 ...
$ EURJPY     : num  130 130 130 130 130 ...
$ EURCAD     : num  1.52 1.52 1.52 1.52 1.52 ...

$ d1_AUDUSD  : num  0.010401 0.000505 -0.001818 -0.003433 -0.008583 ...
$ d1_CHFJPY  : num  -0.01497 0.00444 -0.00186 -0.02309 -0.00823 ...
$ d1_USDCAD  : num  0.008585 0.003389 -0.012832 0.000136 0.002982 ...
$ d1_GBPCHF  : num  -0.000685 0.001746 0.000651 0.024858 -0.009416 ...
$ d1_EURGBP  : num  0.0031 0.00161 -0.00194 -0.01042 0.01003 ...
$ d1_USDJPY  : num  -0.02035 0.00201 -0.00654 0.00558 -0.00933 ...
$ d1_GBPUSD  : num  0.00707 0.00355 0.00439 0.00204 -0.00697 ...
$ d1_EURUSD  : num  0.01916 0.0093 0.00404 -0.01876 0.00905 ...
$ d1_EURCHF  : num  0.0116 0.01463 -0.00732 0.04675 0.01089 ...
$ d1_USDCHF  : num  -0.01705 -0.00415 -0.00933 0.05219 -0.00553 ...
$ d1_GBPJPY  : num  -0.00779 0.00362 -0.00101 0.00484 -0.01057 ...
$ d1_EURJPY  : num  -0.00941 0.00844 -0.0045 -0.00668 -0.00417 ...
$ d1_EURCAD  : num  0.02362 0.01077 -0.01055 -0.01408 0.00971 ...
$ d2_AUDUSD  : num  0.000909 -0.001313 -0.001313 -0.012017 -0.012017 ...
$ d2_CHFJPY  : num  -0.02594 0.00258 0.00258 -0.03132 -0.03132 ...
$ d2_USDCAD  : num  0.01387 -0.00944 -0.00944 0.00312 0.00312 ...
$ d2_GBPCHF  : num  0.0103 0.0024 0.0024 0.0154 0.0154 ...
$ d2_EURGBP  : num  -0.000776 -0.000332 -0.000332 -0.000388 -0.000388 ...
$ d2_USDJPY  : num  -0.01857 -0.00453 -0.00453 -0.00375 -0.00375 ...
$ d2_GBPUSD  : num  0.00771 0.00794 0.00794 -0.00493 -0.00493 ...
$ d2_EURUSD  : num  0.0123 0.0133 0.0133 -0.0097 -0.0097 ...
$ d2_EURCHF  : num  0.03676 0.00732 0.00732 0.05764 0.05764 ...
$ d2_USDCHF  : num  0.00599 -0.01348 -0.01348 0.04666 0.04666 ...
$ d2_GBPJPY  : num  -0.00616 0.0026 0.0026 -0.00572 -0.00572 ...
$ d2_EURJPY  : num  -0.01205 0.00394 0.00394 -0.01085 -0.01085 ...
$ d2_EURCAD  : num  0.023928 0.000221 0.000221 -0.004371 -0.004371 ...
$ d4_AUDUSD  : num  -0.000404 -0.000404 -0.000404 -0.029385 -0.029385 ...
$ d4_CHFJPY  : num  -0.0234 -0.0234 -0.0234 -0.0327 -0.0327 ...
$ d4_USDCAD  : num  0.00443 0.00443 0.00443 0.00122 0.00122 ...
$ d4_GBPCHF  : num  0.0127 0.0127 0.0127 0.0255 0.0255 ...
$ d4_EURGBP  : num  -0.00111 -0.00111 -0.00111 -0.01496 -0.01496 ...
$ d4_USDJPY  : num  -0.0231 -0.0231 -0.0231 0.0175 0.0175 ...
$ d4_GBPUSD  : num  0.0156 0.0156 0.0156 -0.0157 -0.0157 ...
$ d4_EURUSD  : num  0.0256 0.0256 0.0256 -0.0601 -0.0601 ...
$ d4_EURCHF  : num  0.0441 0.0441 0.0441 0.0277 0.0277 ...
$ d4_USDCHF  : num  -0.00749 -0.00749 -0.00749 0.09459 0.09459 ...
$ d4_GBPJPY  : num  -0.0035602 -0.0035602 -0.0035602 0.0000957 0.0000957 ...
$ d4_EURJPY  : num  -0.00811 -0.00811 -0.00811 -0.02169 -0.02169 ...
$ d4_EURCAD  : num  0.0241 0.0241 0.0241 -0.0445 -0.0445 ...
$ d1_AUDUSD_f: Factor w/ 2 levels "-1","1": 2 1 1 1 1 1 1 1 1 2 ...
$ d1_CHFJPY_f: Factor w/ 2 levels "-1","1": 2 1 1 1 2 1 1 2 2 2 ...
$ d1_USDCAD_f: Factor w/ 2 levels "-1","1": 2 1 2 2 1 1 1 2 2 1 ...
$ d1_GBPCHF_f: Factor w/ 2 levels "-1","1": 2 2 2 1 2 2 1 2 1 1 ...
$ d1_EURGBP_f: Factor w/ 2 levels "-1","1": 2 1 1 2 1 1 1 1 2 2 ...
$ d1_USDJPY_f: Factor w/ 2 levels "-1","1": 2 1 2 1 2 2 1 2 2 2 ...
$ d1_GBPUSD_f: Factor w/ 2 levels "-1","1": 2 2 2 1 1 1 1 1 1 2 ...
$ d1_EURUSD_f: Factor w/ 2 levels "-1","1": 2 2 1 2 1 1 1 1 2 2 ...
$ d1_EURCHF_f: Factor w/ 2 levels "-1","1": 2 1 2 2 1 1 1 1 1 2 ...
$ d1_USDCHF_f: Factor w/ 2 levels "-1","1": 1 1 2 1 2 2 2 2 1 1 ...
$ d1_GBPJPY_f: Factor w/ 2 levels "-1","1": 2 1 2 1 2 1 1 2 1 2 ...
$ d1_EURJPY_f: Factor w/ 2 levels "-1","1": 2 1 1 1 2 1 1 1 2 2 ...
$ d1_EURCAD_f: Factor w/ 2 levels "-1","1": 2 1 1 2 1 1 1 2 2 2 ...
$ d2_AUDUSD_f: Factor w/ 2 levels "-1","1": 1 1 1 1 1 1 1 2 2 2 ...
$ d2_CHFJPY_f: Factor w/ 2 levels "-1","1": 2 1 1 1 1 1 1 2 2 2 ...
$ d2_USDCAD_f: Factor w/ 2 levels "-1","1": 1 2 2 1 1 2 2 2 2 1 ...
$ d2_GBPCHF_f: Factor w/ 2 levels "-1","1": 2 2 2 2 2 1 1 1 1 1 ...
$ d2_EURGBP_f: Factor w/ 2 levels "-1","1": 1 1 1 1 1 1 1 2 2 1 ...
$ d2_USDJPY_f: Factor w/ 2 levels "-1","1": 1 1 1 2 2 2 2 2 2 1 ...
$ d2_GBPUSD_f: Factor w/ 2 levels "-1","1": 2 1 1 1 1 1 1 2 2 2 ...
$ d2_EURUSD_f: Factor w/ 2 levels "-1","1": 2 1 1 1 1 1 1 2 2 2 ...
$ d2_EURCHF_f: Factor w/ 2 levels "-1","1": 2 2 2 1 1 1 1 1 1 1 ...
$ d2_USDCHF_f: Factor w/ 2 levels "-1","1": 1 2 2 2 2 2 2 1 1 1 ...
$ d2_GBPJPY_f: Factor w/ 2 levels "-1","1": 2 1 1 2 2 1 1 2 2 2 ...
$ d2_EURJPY_f: Factor w/ 2 levels "-1","1": 2 1 1 1 1 1 1 2 2 1 ...
$ d2_EURCAD_f: Factor w/ 2 levels "-1","1": 2 1 1 1 1 1 1 2 2 2 ...
$ d4_AUDUSD_f: Factor w/ 2 levels "-1","1": 1 1 1 1 1 1 1 2 2 2 ...
$ d4_CHFJPY_f: Factor w/ 2 levels "-1","1": 1 1 1 2 2 2 2 2 2 2 ...
$ d4_USDCAD_f: Factor w/ 2 levels "-1","1": 2 2 2 2 2 2 2 1 1 1 ...
$ d4_GBPCHF_f: Factor w/ 2 levels "-1","1": 2 2 2 1 1 1 1 1 1 1 ...
$ d4_EURGBP_f: Factor w/ 2 levels "-1","1": 1 1 1 1 1 1 1 1 1 1 ...
$ d4_USDJPY_f: Factor w/ 2 levels "-1","1": 2 2 2 2 2 2 2 2 2 2 ...
$ d4_GBPUSD_f: Factor w/ 2 levels "-1","1": 1 1 1 1 1 1 1 2 2 2 ...
  [list output truncated]

Among the listed target variables, only two target variables d4_EURUSD_f and d4_USDCHF_f have predictors that have predictive power for these target variables. All other target variables cannot be predicted 4 steps ahead on the listed predictors.

Predicting 1 step ahead and 2 steps ahead looks much better

 

The up trend is the confidence of the vast majority of participants in the fall, which is supported by the deal...

support is the range in which the vast majority of participants shorted, but the price did not fall, but rather rose. When the price returns to the same range the rest of the scared shorts will aggressively close their shorts at about zero, thus making a mini upward rally.

 
SanSanych Fomenko:

Here is a list of predictors and target variables, which factors, shifted by the appropriate number of bars..........................

..........................Predicting 1 step and 2 steps ahead looks much better

How much better? what are the results?

 
mytarmailS:

How much better? What are the results in general?

There is some abstract measure:

= 1 is 100% predictive ability

If > 10, then I believe the predictor has no predictive ability.

For the listed 4-step-ahead targets, the predictors have a measure of 7 to 9. For the 1-step-ahead prediction, there are predictors with a measure of just over two.

PS.

On this thread I have repeatedly called for dealing specifically with the predictive ability of predictors. Let's not forget: "garbage in, garbage out. And no model can change that.

 
SanSanych Fomenko:

There is some abstract measure:

specifically acuracy at MO what on OOS ?

why these abstractions...

 
mytarmailS:

What exactly is the MO's acuracy on the OOS?

why these abstractions...

A class is predicted. Prediction error is 25 to 30%, and it is the same in training and out of sample. the model is NOT retrained.

PS

I've written all this many, many times.

 
SanSanych Fomenko:

Let's not forget: "garbage in, garbage out. And no model can change that.

Sometimes a combination of garbage predictors and the right model suddenly starts predicting better than those predictors by themselves.

Here's an example of training data from MO competition numer.ai -https://api.numer.ai/competitions/current/dataset(table numerai_training_data.csv in the archive). There are 50 predictors there, and all methods of evaluating them that I tried say that it's garbage. But by trying different combinations of them and different models it's possible to get prediction accuracy >50% on training and validation.

 
Dr.Trader:

Sometimes a combination of junk predictors and the right model suddenly starts predicting better than those predictors by themselves.

Here's an example of training data from the MO competition numer.ai -https://api.numer.ai/competitions/current/dataset(table numerai_training_data.csv in the archive). There are 50 predictors there, and all methods of evaluating them that I tried say that it's garbage. But by trying different combinations of them and different models - you can get prediction accuracy >50% on training and validation.

1. You need to initially take two separate files: one for training-testing-validation, and one for checking the model created. The error on all four samples should be about the same.

2. Naturally, our whole activity is to manipulate the original quotient and obtain a new, derived from the original, precursor that will have predictive power. I'm not discussing where the original set of predictors came from - that's a problem in its own right.

Reason: