Discussion of article "Random Forests Predict Trends" - page 8

[Alexey Navoykov]

Demi:

no, it is not clear - I don't have a cat and I doubt that the temperature of any pet is relevant to forex information.

By the way, I give 100% that the dynamics of the cat's temperature will have a non-zero correlation with the price series of a financial instrument

Well, earlier you claimed that all data has predictive power. Of course we are talking about predicting financial markets.

If the computing power of our computers allowed us to use all available information, including the cat's temperature, it would simply be eliminated in the calculation process. But since resources are limited, we have to use our own head.As for the correlation, the fact that it is non-zero is a given, zero can rarely happen at all. But it is almost zero. In general, buy a cat, put a thermometer on it and check it )) Maybe you will be lucky and it will make you rich ).

[Дмитрий]

meat:

Now, earlier you said that all data has predictive power. Of course we're talking about predicting financial markets.

If the computing power of our computers allowed us to use all available information, including the cat's temperature, it would simply be discarded in the calculation process. But since resources are limited, we have to use our own head.About the correlation, the fact that it is non-zero - it goes without saying, zero in general can rarely happen. But almost zero. In general, buy a cat, put him a thermometer and check it )) Maybe you will be lucky and he will make you rich )

you are lying - I quote myself"I give you the answer to your question - there is some "predictive power" in all data. So? There is some sort of information in all forex data.".

If you have a set of hundreds of thousands of observations - the question of computer power would be relevant.

[Alexey Navoykov]

Demi:

you are lying - I quote myself"I give you the answer to your question - there is some "predictive power" in all data. So? There is some sort of information in all forex data.".".

meat:

you previously claimed that there is predictive power in all data

And what is the lie?

[Дмитрий]

meat:

And what's the lie?

I decoded myself in the second sentence.

[Alexey Navoykov]

Demi:
I decoded myself in the second sentence.

I.e. information = predictive power? How can you be so sure of that? If I tell you some of the "forex data", you can predict the price by it?

There are not only forex, but also stock markets, commodity markets and many other things.... Everything is interconnected.

[Dmitriy Skub]

faa1947:

I have a dozen customers here. Before communication with me all were joyful and cheerful, but now they are sad and pensive.

Do not multiply the sadness)))

[Vladimir]

faa1947, please show how your model works on the example below. The first column is the modelled series, the 2nd and 3rd columns are predictors. What is the predictive power of these predictors?

-0.35742	0.461646	-0.81907
0.431277	0.004545	0.426731
-0.00018	-0.4037	0.403528
-0.08575	0.90851	-0.99426
0.773826	0.008975	0.764852
0.426905	-0.96485	1.391757
0.253233	0.487955	-0.23472
0.20994	0.880554	-0.67061
-0.09929	0.160276	-0.25956
0.332911	-0.08736	0.420268
0.032258	0.360106	-0.32785
0.253027	-0.06859	0.321621
-0.66668	-0.54985	-0.11683
-0.5476	-0.13231	-0.41529
-0.75652	0.536488	-1.29301
-0.66109	-0.87314	0.212052
-0.09993	-0.86293	0.763
0.014625	0.715032	-0.70041
-0.48345	-0.62666	0.143206
-0.03596	0.935653	-0.97161
-0.17023	0.678024	-0.84826
0.293376	0.079529	0.213847
0.002922	0.754594	-0.75167
0.329194	-0.05535	0.384546
0.639423	-0.41358	1.053007
0.431631	-0.60334	1.034973
0.59125	0.497989	0.093262
0.266098	-0.79645	1.062549
-0.02624	0.643164	-0.6694
0.055014	-0.46214	0.517154
0.436132	-0.89992	1.336052
-0.30143	0.628619	-0.93005
-0.12168	0.886467	-1.00814
-0.10851	-0.0507	-0.0578
-0.74573	-0.50921	-0.23653
-0.574	0.244825	-0.81883
-0.87313	0.336188	-1.20932
-0.00344	0.117363	-0.1208
-0.20265	0.424326	-0.62697
0.177873	-0.17967	0.357541

[Yury Reshetov]

gpwr:

What is the predictive power of these predictors?

Apparently, the numbers are not random. I checked it very simply: I generated three rows of 40 rows randomly and applied neuron to them. On random data, the generalisation ability is noticeably worse than on the above-mentioned data.

[Vladimir]

Reshetov:
Apparently, the numbers are not random. I checked it very simply: I generated three rows of 40 rows randomly and applied neuronka to them. On random data the generalisation ability is noticeably worse than on the above-mentioned data.

I would be interested in different universal algorithmic approaches to determine the predictive ability of these two predictors. Econometrics, neuronics and deep networks are welcome. Show us what they can do. You can show either some parameter characterising such predictive ability of these predictors (correlation coefficient, mutual information, RMS, and other inventions) or show comparison of model output and simulated series.

[Yury Reshetov]

gpwr:
I would be interested in different universal algorithmic approaches to determine the predictive power of these two predictors. Econometrics, neuronics and deep networks are welcome. Show us what they can do. You can show either some parameter characterising such predictive ability of these predictors (correlation coefficient, mutual information, RMS, and other inventions) or show comparison of model output and simulated series.

1. The term "predictive ability" is the stuff of fortune tellers, psychics, shamans, and other econometricians. In machine learning one can calculate generalisation ability, and even then only approximately
2. There is not enough data, i.e. we are dealing with Small Data (only 40 examples), and therefore the estimates of generalisation ability may be overestimated, i.e. a finger in the sky.
   

I brought the sample to a form suitable for binary classification, i.e. to calculate the dependent variable to see if it is above zero or below (CSV file in the attached archive), searched with libVMR and got this model:

/**
* The quality of modelling:
*
* TruePositives: 9
* TrueNegatives: 11
* FalsePositives: 0
* FalseNegatives: 0
* Sensitivity of generalization abiliy: 100.0%
* Specificity of generalisation ability: 100.0%
* Generalization ability: 100.0%
*/
double x0 = 2.0 * (v0 + 0.96485) / 1.900503 - 1.0;
double x1 = 2.0 * (v1 + 1.00814) / 2.399897 - 1.0;

y = 0.12981203254657206 + 0.8176828303879957 * x0 + 1.0 * x1 -0.005143248786272694 * x0 * x1;

---

The secret of the "high generalisability" of your sample is revealed: the value of the first column is the sum of the values of the other two columns.