TheEcologist (09-17-2013)
Today a labmate asked the following question: if we have three random variables x, y, z, and we know the correlations and , what constraints if any does this place on the correlation ?
At the time I reflexively answered that the remaining correlation must be the product of the two known correlations. Which of course is totally wrong. I think I was getting some mental interference from some of the equations for simple mediation floating around in my head. Anyway, after thinking about it for a while I have come up with a convincing geometric argument for what the constraints actually are. I have also verified that my answer agrees with a more complicated-looking answer to this question that I found elsewhere online.
Because I ending up spending a lot of time on this and I thought some of you would find the results interesting, I thought I would share my work here. Comments are welcome!
Okay. So imagine our variables x, y, z as vectors in n-dimensional space. The Pearson correlation coefficient between any two of these variables can be interpreted as the cosine of the angle between the corresponding vectors. This is an interesting and well-known geometric fact about correlation coefficients.
So now imagine that the x and y vectors are fixed (and hence so is their correlation), but that the vector z is free to vary so long as is constant. This constraint on means that the set of possible z vectors will form a sort of "cone" around the y vector, as in the following image:
Now it is intuitively obvious (I know this is a sneaky phrase, but that's why I call this just an "argument" and not a "proof") that the two possible z vectors that will lead to the minimum/maximum values of are the z vectors that lie on the same plane as the x and y vectors. This leads to the following expression for the minimum/maximum values of given and :
One notable result following from this is that if x is orthogonal to y and y is orthogonal to z, then there is no constraint on , it can be anywhere from -1 to +1. But under any other circumstances, fixing and will place some constraint on the range of .
Okay, now for the verification part, which requires a bit of math.
So in this stats.stackexchange.com thread (LINK), it is stated that the three correlations must satisfy
The reasoning here being "because this is the determinant of the correlation matrix and it cannot be negative." Anyway, this can be viewed as a quadratic inequality in , already in standard form:
So if we apply the quadratic formula and simplify the result, we get the following for the minimum/maximum values of :
Now taking my answer and applying the trig identity we get
Now applying the identity we get
Which is the answer we got from the stackexchange post. So our simpler, geometrically based answer agrees with the more conventional answer that is harder to understand.
In God we trust. All others must bring data.
~W. Edwards Deming
TheEcologist (09-17-2013)
Nice work. I would argue against your last statement that the conventional answer is "harder to understand". I will admit that it's harder to interpret geometrically - but algebraically we know that that constraint (positive determinant) must hold. So that argument is really just mindlessly going through the motions starting from that one fact that we know.
To get honest that alternative approach is ultimately where I ended up when I was thinking about the problem. My first thought was just to find the values that makes the correlation matrix positive semi-definite. After *a little* bit of work we can show that this is equivalent to the determinant criteria in this case. I like your approach more - geometric approaches always seem more creative.
I don't have emotions and sometimes that makes me very sad.
Here we just view the covariance to be an inner product such that the correlation actually is just the cosine of included angle . It is always interesting to have this geometric interpretation in our mind. And actually there are quite a lot of tools in Hilbert space can be used in probability context.
Hi,
I have a somewhat similar and easier question about relationships between correlations that I can't figure out, would any of you be able to put me on the right track?
I have two variables, x and z, for which Corr(x,z) > 0. I define a new variable y = x - z. I want to know what are the constraints on Corr(x,y) given the above fact about Corr(x,z). Intuitively it seems like it must hold that Corr(x,y) >= Corr(x,z). I can't show this mathematically however.
Any ideas?
well... i don't think that's necessarily true. if x=z then your first condition of Cor(x,z)>0 holds (because Cor(x,z) = Cor(z,z) = 1) but then you would have Cor(x,y) = Cor(x,x-z)=Cor(x,z-z)=Cor(x,0)=0 and you would end up with 0>=1, which is obviously wrong.
i feel like you would need to tell us more about the relationship between x, y and z to come up with suitable boundaries.
for all your psychometric needs! https://psychometroscar.wordpress.com/about/
Jake (05-18-2015)
I replied to your other thread about this.
In God we trust. All others must bring data.
~W. Edwards Deming
Tweet |