Relationship between Pearson correlation and variance in regression model

**_joey** · 08-12-2012 08:50 PM

If the correlation between X and Y, say, is 0.5, then the standard deviation of the differences between the Y values and their predicted values on the regression line is closest to which of the following fractions of the standard deviation of the Y values: 0.25, 0.5, 0.75 or 0.87

That's a homework question and I am asking for a solution. I am looking for a guidance in the right direction. I am looking for a formula that shows relationship between pearson correlation and variance of the residuals.

Thanks!

**Dason** · 08-12-2012 09:08 PM

Have you worked with $R^2$ before? Do you know how to interpret $R^2$ .

**_joey** · 08-12-2012 09:19 PM

Originally Posted by Dason

Have you worked with $R^2$ before? Do you know how to interpret $R^2$ .

Thanks Dason.

I came across it in statistical output but I haven't interpreted. In Wikipedia looking at the formula R^2 is square of sample correlation. It is also R^2=1-SSerr/SStot

**Dason** · 08-12-2012 09:22 PM

Do you know what SSerr and SStot represent? They're highly related to what you're interested in.

**_joey** · 08-12-2012 09:23 PM

I can't see it. The formulae involve the sum of square. The question ask about standard deviation.

**Dason** · 08-12-2012 09:25 PM

Do you not know how those quantities are defined or you don't see how they're related to what you're interested in? Either way you should find out how they're defined and consider trying to formalize what it is exactly what you're looking for.

**_joey** · 08-12-2012 09:32 PM

Originally Posted by Dason

Do you not know how those quantities are defined or you don't see how they're related to what you're interested in? Either way you should find out how they're defined and consider trying to formalize what it is exactly what you're looking for.

The definitions are in the formulae? No?

My solution as follows: R^2=0.5, then approximately fifty percent of the variation in the response variable can be explained by the explanatory variable. The remaining fifty percent can be explained by inherent variability.

I see I am not getting anywhere here. I will contact my lecturer. Thanks for your time.

**Dragan** · 08-12-2012 09:45 PM

Originally Posted by _joey

My solution as follows: R^2=0.5,.

I see I am not getting anywhere ....

Joey, in your first post you defined the Pearson correlation to be r = 0.5. So, how can r^2 be 0.5.

**_joey** · 08-12-2012 09:51 PM

Originally Posted by Dragan

Joey, in your first post you defined the Pearson correlation to be r = 0.5. So, how can r^2 be 0.5.

well, r^2=1-SSer/SStot => 0.25 = 1 - 0.75. I still can't see the connection between pearson correlation and residual standard deviation in terms of fractions.
SS involve sums of squares.

**Dason** · 08-12-2012 09:56 PM

And standard deviation involves sums of squares (that have been scaled). So it's quite possible that these things are highly related... right?

Like I said before - I think you should actually try to figure out how to formalize what it is you're looking for. What I mean is write down a mathematical statement of what quantity it is you're looking for. Then see if you can relate that to the formula you have for r^2.

A lot of these problems are just doing that - playing around with these quantities and manipulating them until you can see how everything is connected. It might take some time and it might be really frustrating but you won't learn anything if you don't play around.

**_joey** · 08-12-2012 10:00 PM

Originally Posted by Dason

right?.

Yeah, right!

Except that I never saw standard deviations being scaled.

**Dason** · 08-12-2012 10:13 PM

I wasn't saying that we're scaling the standard deviation. I was saying that the standard deviation is a scaled form of the sums of the squares. What I mean is that the sample estimate of the standard deviation is just the sum of squares divided by n-1.

So - yes - you have seen standard deviations being scaled - because the sums of squares can just be seen as a scaled version of the standard deviation as well. But that is somewhat irrelevant. My original point still stands. I think you should actually try to figure out how to formalize what it is you're looking for. What I mean is write down a mathematical statement of what quantity it is you're looking for. Then see if you can relate that to the formula you have for r^2.

**_joey** · 08-12-2012 10:26 PM

Originally Posted by Dason

I wasn't saying that we're scaling the standard deviation. I was saying that the standard deviation is a scaled form of the sums of the squares. What I mean is that the sample estimate of the standard deviation is just the sum of squares divided by n-1.

So - yes - you have seen standard deviations being scaled - because the sums of squares can just be seen as a scaled version of the standard deviation as well. But that is somewhat irrelevant. My original point still stands. I think you should actually try to figure out how to formalize what it is you're looking for. What I mean is write down a mathematical statement of what quantity it is you're looking for. Then see if you can relate that to the formula you have for r^2.

There may or there may not be a link between the question and R^2. You are not sure yourself, hence you are guessing, bringing irrelevant discussion (scaling) or withholding some information while suggesting the TS should do his homework himself.
I wasted an hour here and I think I should start looking for another forum with homework section

Thanks anyway!

**Dason** · 08-12-2012 10:35 PM

Originally Posted by _joey

There may or there may not be a link between the question and R^2.

You want me to be more clear? There is a link.

You are not sure yourself,

I don't know what would give you that idea. Was it because I'm not doing your homework directly for you and instead am trying to steer you toward the correct answer in a manner that you actually learn and understand the material?

hence you are withholding some information while suggesting the TS should do his homework himself.

Of course I'm withholding some information. I know the answer! I have to withhold information if I want you to actually learn something. I seem to recall somebody saying...

Originally Posted by _joey

I am looking for a guidance in the right direction

so either you were lying or you're just not satisfied with the guidance we've given. I'm not sure if you've read our homework help policy and our general posting guidelines but they should give you a good idea of what to expect here (and you've been a member for some time so I would hope you've stumbled upon those threads at some point).

I thought I was helping. I was hoping I was helping. You didn't seem to be getting it though and didn't seem to be trying what we were suggesting. I wasn't suggesting random crap for no reason. I really thought it would help you see the connection that you should see. It takes work though. If you aren't willing to put in the amount of work we ask for then maybe you should find a different place to ask questions. That isn't what I want though. I want you to stay and feel free to ask questions. Just know that we have rules and guidelines and for the most part we know what we're doing when it comes to this simple stuff (it might not seem simple to you but a lot of us have quite a bit of experience so it is relatively simple for a few of us here).

PS - I notice you added a bit about scaling into your last post. All I was trying to do was make you see that standard deviation and sums of squares are in fact related since they're just scaled forms of one another. You said you didn't see how the sums of squares and standard deviation were related so I was trying to show you that they were.

**Dragan** · 08-12-2012 11:33 PM

Originally Posted by _joey

, bringing irrelevant discussion (scaling)
Thanks anyway!

It's not irrelevant, joey. Consider the following well known relationship with the Pearson correlation and the standard deviations between the predicted scores and the actual scores of Y (the dependent variable):

$r=\frac{S_{\hat{y}}}{S_{y}}$

which is all we need.

In other words, you should be able to do the following:

$r=\frac{S_{\hat{y}}}{S_{y}}$

$rS_{y}=S_{\hat{y}}$

$r^{2}S_{y}^{2}=S_{\hat{y}}^{2}$

$r^{2}S_{y}^{2}=\frac{SS_{reg}}{n-1}$

$\left ( n-1 \right )r^{2}S_{y}^{2}=SS_{reg}=SS_{tot}-SS_{e}$

$SS_{tot}-\left ( n-1 \right )r^{2}S_{y}^{2}=SS_{e}$

dividing through both sides by n - 1 and simplifying gives:

$S_{y}^{2}-r^{2}S_{y}^{2}=S_{e}^{2}$

and thus,

$S_{y}^{2}\left ( 1-r^{2} \right )=S_{e}^{2}$

Hence,

$S_{y}\sqrt{1-r^{2}}=S_{e}$

and we're done.

Assuming, r=0.5, the answer is 0.87 (to two decimals). Now how difficult is this, Joey?

Thread: Relationship between Pearson correlation and variance in regression model

Thread Tools

Display

Relationship between Pearson correlation and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

Re: Relatioship between pearson corr and variance in regression model

The Following 2 Users Say Thank You to Dason For This Useful Post:

Re: Relatioship between pearson corr and variance in regression model

Posting Permissions