Zero sample correlation does not necessarily mean there is no linear relationship - page 8
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Everywhere I read, they say that a zero sample correlation means that there is no linear (usually forget the word linear) relationship in that sample.
If only in simplified form, then yes, that is what it says. And that is, in general, correct. And the subtleties are of no use to the general public.
hrenfx:
Example of two graphs with zero MO, variance one and correlation zero. That is, the correlation in this case is the sum of the products of the BP terms divided by the length of the BP.
Here are my graphs. This is the data.
And here's Pearson, trivially calculated in Excel. With one caveat, the calculations were done in a sliding window.
As you can see, in the crudest approximation, the coefficient varies over time. It seems to me that it was you, who I pointed out before, that there are problems of stationarity in the time series of prices. And about the need to be careful when using statistical methods not designed for non-stationary data. (Although, the problem here is somewhat more complex than non-stationarity.)
hrenfx:
In fact, there is always a linear relationship between any two random variables on a finite sample.
Be careful about interpreting correlations close to zero.
In fact, there is a linear relationship between two random variables on those two series. It may not be there on the others. That's one. Two, the coefficient, like any decent estimate of a random variable, has an area of confidence. Anyway, your sentiments don't matter.
In general, be careful what you say about things you don't fully understand.
As can be seen, in the crudest approximation, the coefficient varies over time.
This is obvious. As well as the fact that the dynamics of the coefficient will depend on the size of the sliding window.
I believe it was to you that I already pointed out the stationarity problems in time series prices. And about the need to be careful when using statistical methods not designed for non-stationary data.
No statistical methods are used at all. Non-stationarity has nothing to do with it.
In fact, there is a linear relationship between the two random variables on those two series. It may not exist on the others. That's one. Two, the coefficient, like any decent estimate of a random variable, has an area of confidence. Anyway, your sentences don't make any sense.
How do you even understand linear correlation? I've already written that in an academic sense it's a measure of the angle between vectors. And that's a poor definition when it comes to interconnection.
There is no linear relationship only when the variance of one of the vectors is zero. In all other cases there is a relationship.
And once again, we are talking about estimates on samples, not theoretical infinite BPs.
This is obvious. So is the fact that the dynamics of the coefficient will depend on the size of the sliding window.
From what you've written and drawn here, I haven't seen that this is the case. It's about obviousness. Well, understanding the meaning of stationarity is not as straightforward as you are trying to make it out to be, through the size of the window.
No statistical methods are used at all. Non-stationarity has nothing to do with it.
Actually, the correlation coefficient belongs to a section of mathematical statistics called correlation analysis. And it was invented by mathematical statisticians. So as soon as you try to calculate a correlation coefficient, you are automatically using statistical methods. And you have to consider all the limitations of those methods.
How do you even understand the linear relationship? I've already written that in an academic sense it's a measure of the angle between vectors. And it's a poor definition when it comes to intercoupling.
There is no linear relationship only when the variance of one of the vectors is zero. In all other cases there is a relationship.
And once again, we are talking about estimates on samples, not theoretical infinite BPs.
Not exactly like that. And it was explained above why not. In statistics, under certain conditions, coefficient =0 and coefficient =0.7 can mean the same thing - no or weak connection.
From what you've written and drawn here, I haven't seen that this is the case. It's about obviousness. Well, understanding the meaning of stationarity is not as simple as you are trying to make it out to be, through the size of the window.
You are, for some reason, making it up to me. I won't use a term I don't understand. And a definition of which I don't know.
Actually, correlation coefficient belongs to a section of mathematical statistics called correlation analysis. And it was invented by mathematical statisticians. So as soon as you try to calculate correlation coefficient you automatically use statistical methods. And you have to consider all the limitations of those methods.
Also familiar with correlation and regression analysis. I don't use any statistical methods. I think of correlation coefficient as the simplest thing that comes to mind when you need to estimate a relationship. This is school level. And without knowledge of Pearson, I got to that almost as soon as I thought about correlations.
Not really. And it was explained above why not. In statistics, under certain conditions, coefficient =0 and coefficient =0.7 can mean the same thing - the absence or weakness of a relationship.
My conclusion is that correlation (Pearson's coefficient) is a shitty indicator of a linear relationship in a sample. Not only does it not show a direct correlation, but it also lies.
Do you call this a criticism of the Pearson coefficient? I'm criticising smart people who misinterpret it by talking about the presence/absence of a relationship, and not even understanding what a linear relationship is.
Speaking of the inconsistency of my methods, you should at least mention one. Pearson methods have not been discussed in this thread.
Also, I haven't read any mathematical statistics books. The correlation and regression analysis was studied, when all this simple toolkit written by mathematicians in a complicated language was already functional in my MQL4. And you don't have to be a jack-of-all-trades to understand 90% of what is written in books about correlation and regression. I mean the practical part, not the theoretical part that takes up most of the books.
Do you call this a criticism of the Pearson coefficient? I'm criticising smart people who misinterpret it by talking about the presence/absence of a relationship, and not even understanding what a linear relationship is.
Yes. What's more, you are criticising C. Pearson in a barefaced way. Write more clearly so that it is unambiguously clear to everyone what exactly it is that you are challenging.
hrenfx:
Speaking of the inconsistency of my methods, you'd at least mention one. Pearson methods have not been discussed in this thread.
The pairwise linear correlation coefficient is what you were talking about.
And as for the methods, you have amazing results in your topicstart. And by pointing your finger at them you draw surprising conclusions. How it can be regarded, except as failure of your methods - I refuse to understand.
hrenfx:
Also, I haven't read a book on mathematical statistics.
Give my regards to Mitrofanushka.
hrenfx:
I've learned correlation and regression analysis when all those simple tools, written by mathematicians in a complicated language, were already functional in my MQL4. And you don't have to be a jack-of-all-trades to understand 90% of what is written in books about correlation and regression. I mean the practical part, not the theoretical part which takes up most of the books.
What you have shown here - clearly shows that you have not studied anything properly. Formulas in section correlation Pearson really not difficult, but the fact that you are able to add figures on formulas, it at all doesn't mean that you are able to use the given mat.apparatus correctly. And your reasoning shows that there is something wrong with your understanding.
Do you call this a criticism of the Pearson coefficient? I'm criticising clever people who misinterpret it by talking about the presence/absence of a relationship, and not even understanding what a linear relationship is.
...
Don't you honestly think that none of present here on a forum for about 5-8 years, never guessed to build CC, that nobody heard about Pearson and even if heard, can't program it (calculate), etc.
Well listen (read), here in this thread has already checked many. And they tell you about the same thing, in different words, but are trying to explain something. Yes, I too have recently frequently argued with HideYourRichess, but I can assure you he is very competent and expert in his work. Like almost everyone who here marked the branch. Yes we may not agree in views, but a respectful conversation helps to gain knowledge, sometimes if the opponents are attentive to each other, in an argument the truth is born ...
It seems to me that you want to explore (build) something, but can not explain it in any way. Many people don't understand you, you use commonly known terms to explain, but you use them in the wrong way. Try explaining it with formulas. Just write it in an understandable way. And explain what and how you want to calculate or have calculated.
Don't anger God (even though I don't believe in him), but to accuse everyone indiscriminately that they use Pearson incorrectly, calculate it, don't understand it, is too presumptuous...
hrenfx, I can't understand it, have you read too many books?
What the hell is logarithm, what buckets and kilograms????? What other "interpretation" of the correlation coefficient than the well-known one that has been around for a hundred years? My advice to you - get some sleep first, and then start learning the math from scratch. Privalov wrote a script, the results are consistent with Matkad's. I wrote the script without looking at the others, I compared the results and they are the same as Beer's and Matkad's. One hundred and fifty people have already written this QC a hundred and fifty times - and all the results are the same. So why would everyone suddenly rush to rewrite their programs, suddenly finding out that someone has their own interpretation of Pearson's QC?
.........
If the sample seems small, let's take something bigger from the correlation table:
.........