+ Reply to Thread
Results 1 to 9 of 9

Thread: Writing R^2 strictly in terms of simple correlations among variables

  1. #1
    Cookie Scientist
    Points: 13,431, Level: 75
    Level completed: 46%, Points required for next Level: 219
    Jake's Avatar
    Location
    Austin, TX
    Posts
    1,293
    Thanks
    66
    Thanked 584 Times in 438 Posts

    Writing R^2 strictly in terms of simple correlations among variables




    Say we have the following regression

    Y_i=\beta_0+\beta_1X_{1i}+\beta_2X_{2i}+\epsilon_i

    I want to write the multiple R^2 for this regression strictly in terms of the simple correlations among Y, X_1, and X_2. In the past we have (I think) discussed how to do this in a more general setting using some matrix calculations. What I want to find is the solution in non-matrix form for the special case where we have 2 predictors.

    The top of p. 2 in the following course notes (http://www2.hawaii.edu/~halina/603/603multreg.pdf) suggests that the answer is

    R^2=\frac{r^2_{YX_1}+r^2_{YX_2}-2r_{YX_1}r_{YX_2}r_{X_1X_2}}{1-r^2_{X_1X_2}}

    But I am having some trouble verifying this algebraically. I wonder if one of you could help point me in the write direction for deriving this from scratch.
    “In God we trust. All others must bring data.”
    ~W. Edwards Deming

  2. #2
    TS Contributor
    Points: 22,432, Level: 93
    Level completed: 9%, Points required for next Level: 918
    spunky's Avatar
    Location
    vancouver, canada
    Posts
    2,135
    Thanks
    166
    Thanked 537 Times in 431 Posts

    Re: Writing R^2 strictly in terms of simple correlations among variables

    only the devil and you know why you want to do this FROM SCRATCH... but ok, here are my $0.02.

    so, the formula we're working with is that R-squared in matrix form can be expressed as:

    r_{xy}^{T}R_{xx}^{-1}r_{xy}

    where r_{xy} is the vector of correlations between the dependent variable Y and the predictors X1 and X2 and R_{xx} is the correlation matrix for predictors X1 and X2.

    so the first step is finding the inverse of:

    \left[
  \begin{array}{ c c }
     1 & r_{x_{1}x_{2}} \\
     r_{x_{1}x_{2}} & 1
  \end{array} \right]

    for which we will use the super-duper handy Wolfram Calculator which says that R_{x_{1}x_{2}}^{-1} is:

    \left[
  \begin{array}{ c c }
     \frac{1}{1-r_{x_{1}x_{2}}^{2}} & \frac{r}{r_{x_{1}x_{2}}^{2}-1} \\
     \frac{r}{r_{x_{1}x_{2}}^{2}-1} & \frac{1}{1-r_{x_{1}x_{2}}^{2}}
  \end{array} \right]

    so all that's left to do is just work the convenient matrix multiplication of:

    \left [ r_{x_{1}y}r_{x_{2}y} \right ] \left[
  \begin{array}{ c c }
     \frac{1}{1-r_{x_{1}x_{2}}^{2}} & \frac{r}{r_{x_{1}x_{2}}^{2}-1} \\
     \frac{r}{r_{x_{1}x_{2}}^{2}-1} & \frac{1}{1-r_{x_{1}x_{2}}^{2}}
  \end{array} \right] \left[
  \begin{array}{ c }
     r_{x_{1}y} \\
     r_{x_{2}y} 
  \end{array} \right]

    and after a lot of arduous and tedious matrix multiplications that i will let you thoroughly enjoy, you end up with the desired result.

    just for kicks and giggles, i plugged all of this into my Maple and i get the exact same formula that you posted (but with switched signs for some reason, so i guess i can multiply it times -1 to get exactly what you got). but i can see that Maple inverted R_{x_{1}x_{2}} with the opposite signs that Wolfram did, so that's probably where it came from.

    enjoy!
    for all your psychometric needs! https://psychometroscar.wordpress.com/about/

  3. The Following 2 Users Say Thank You to spunky For This Useful Post:

    Jake (10-18-2014), kiton (11-14-2014)

  4. #3
    Super Moderator
    Points: 13,151, Level: 74
    Level completed: 76%, Points required for next Level: 99
    Dragan's Avatar
    Location
    Illinois, US
    Posts
    2,014
    Thanks
    0
    Thanked 223 Times in 192 Posts

    Re: Writing R^2 strictly in terms of simple correlations among variables

    You can simply use the fact that:

    \beta _{1}=\frac{r_{y1}-r_{r2}r_{12}}{1-r_{12}^{2}}
    and

    \beta _{2}=\frac{r_{y2}-r_{r1}r_{12}}{1-r_{12}^{2}}.

    Substitute these two expressions into the following equation

    R_{Y.12}^{2}=\beta _{1}r_{y1}+\beta _{2}r_{y2}.

    Simplifying the expression above will yield the result you’re looking for.
    Last edited by Dragan; 10-18-2014 at 11:11 AM.

  5. The Following User Says Thank You to Dragan For This Useful Post:

    Jake (10-18-2014)

  6. #4
    Cookie Scientist
    Points: 13,431, Level: 75
    Level completed: 46%, Points required for next Level: 219
    Jake's Avatar
    Location
    Austin, TX
    Posts
    1,293
    Thanks
    66
    Thanked 584 Times in 438 Posts

    Re: Writing R^2 strictly in terms of simple correlations among variables

    Quote Originally Posted by spunky View Post
    just for kicks and giggles, i plugged all of this into my Maple and i get the exact same formula that you posted
    Maple will show the element-wise results of matrix multiplications? I've got to get Maple! I've been looking for a program that would do this. I have a Mathematica license through my school but as far as I can tell it won't do this (or at least I can't figure out how to get it to). I just looked and it seems that my school doesn't have a Maple license... know of any other software that will do this?

    Quote Originally Posted by Dragan View Post
    You can simply use the fact that...
    Thanks Dragan, that is super useful. I did not know those expressions that you posted, but they certainly do make this a simple problem if you know them.
    “In God we trust. All others must bring data.”
    ~W. Edwards Deming

  7. #5
    Devorador de queso
    Points: 95,889, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,937
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Writing R^2 strictly in terms of simple correlations among variables

    If you give a concrete example (this would work) you could probably ask how to do what you want at the mathematica stackexchange site: http://mathematica.stackexchange.com/
    I don't have emotions and sometimes that makes me very sad.

  8. #6
    TS Contributor
    Points: 22,432, Level: 93
    Level completed: 9%, Points required for next Level: 918
    spunky's Avatar
    Location
    vancouver, canada
    Posts
    2,135
    Thanks
    166
    Thanked 537 Times in 431 Posts

    Re: Writing R^2 strictly in terms of simple correlations among variables

    Quote Originally Posted by Jake View Post
    Thanks Dragan, that is super useful. I did not know those expressions that you posted, but they certainly do make this a simple problem if you know them.
    that is NOT fair! i assumed Jake wanted everything from scratch. had i known this, i would have simply posted that formula and not struggle with the element-wise matrix productsss!! all those formulas (and more nuggets of wisdom) are in Cohen's regression bible. seriously, after reading the first few chapters of that book you learn to respect the correlation matrix because of all the stuff you can get from it.

    and yes, Maple *DOES* show you the element-wise product of the matrices. once again, a CANADIAN piece of software beats the more popular AMERICAN alternative (go Canada!!!!)
    for all your psychometric needs! https://psychometroscar.wordpress.com/about/

  9. #7
    TS Contributor
    Points: 22,432, Level: 93
    Level completed: 9%, Points required for next Level: 918
    spunky's Avatar
    Location
    vancouver, canada
    Posts
    2,135
    Thanks
    166
    Thanked 537 Times in 431 Posts

    Re: Writing R^2 strictly in terms of simple correlations among variables

    this is what Maple does for you. if you evaluate the last expression and multiply it times -1/-1 you'd get the same formula

    for all your psychometric needs! https://psychometroscar.wordpress.com/about/

  10. #8
    Cookie Scientist
    Points: 13,431, Level: 75
    Level completed: 46%, Points required for next Level: 219
    Jake's Avatar
    Location
    Austin, TX
    Posts
    1,293
    Thanks
    66
    Thanked 584 Times in 438 Posts

    Re: Writing R^2 strictly in terms of simple correlations among variables

    Quote Originally Posted by spunky View Post
    that is NOT fair! i assumed Jake wanted everything from scratch. had i known this, i would have simply posted that formula and not struggle with the element-wise matrix productsss!!
    Well like I said, it's all very useful, but if I'm being nitpicky I don't really consider either method to be "from scratch" for my purposes, since the first method relies on knowing the matrix expression in the first place, which I don't know how to derive, and the second method relies on knowing those equations which I also don't know how to derive. I was thinking more along the lines of something that started with the fundamental definition of R^2 and worked up from there, but the suggestions given certainly do give some clues about what the full derivation would look like, basically by breaking it down to some smaller and more tractable problems that I'm pretty sure I could solve if I try for a bit.
    “In God we trust. All others must bring data.”
    ~W. Edwards Deming

  11. #9
    TS Contributor
    Points: 22,432, Level: 93
    Level completed: 9%, Points required for next Level: 918
    spunky's Avatar
    Location
    vancouver, canada
    Posts
    2,135
    Thanks
    166
    Thanked 537 Times in 431 Posts

    Re: Writing R^2 strictly in terms of simple correlations among variables


    Quote Originally Posted by Jake View Post
    Well like I said, it's all very useful, but if I'm being nitpicky I don't really consider either method to be "from scratch" for my purposes, since the first method relies on knowing the matrix expression in the first place, which I don't know how to derive, and the second method relies on knowing those equations which I also don't know how to derive. I was thinking more along the lines of something that started with the fundamental definition of R^2 and worked up from there, but the suggestions given certainly do give some clues about what the full derivation would look like, basically by breaking it down to some smaller and more tractable problems that I'm pretty sure I could solve if I try for a bit.
    i've just done this for you. you can check out where my formula (and Dragan's) come from on this blog entry of mine:

    CLICK HERE AND THE ANSWER WILL BE REVEALED
    for all your psychometric needs! https://psychometroscar.wordpress.com/about/

  12. The Following User Says Thank You to spunky For This Useful Post:

    Jake (02-08-2015)

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats