variance of simple linear regression coefficients

#1
Just starting learning statistics and regression so pardon my ignorance.

Around 36th minute of this video there is a discussion related to the variance of linear regression coefficients
I am struggling to understand what is practically means as the least squares method will derive specific values of these parameters which could be treated as constants hence they would not ideally have a variance.
Could someone please elaborate on this. Have spent couple of hours searching various videos/webpages but none seem to provide the detail and dive directly into deriving the formula.
 

Dason

Ambassador to the humans
#3
Sure. When you have a fixed sample you get fixed estimates. That's not what the variance is referring to here. The assumptions of your model probably talk about some random variables and in the case of regression it's typically that the errors are assumed to be i.i.d. N(0, sigma).

So let's say you took another sample of the same size as your original. Would you expect to get the exact same slopes and intercept? Probably not because there is randomness involved. Imagine you do this process (sample and then estimate parameters) over and over. You'll end up with a lot of estimates for the slopes and intercepts. You might even imagine it makes sense to talk about the distribution of those estimates. Let's just talk about the intercept estimate to keep things simple. So if you visualized what your intercept estimate distribution looked like over many samples we would be getting at what is called the sampling distribution of the intercept. It's the variance of *that* distribution that the video is referring to. Now why do we care about this? Because typically we have questions about the *true* intercept (or more likely the *true* slope). By using the sampling distribution we can start to do some statistical tests on some hypothesis we may have about the true parameter (i.e. is the parameter equal to 0 or do we have evidence that it isn't equal to 0?).
 
#5
Sure. When you have a fixed sample you get fixed estimates. That's not what the variance is referring to here. The assumptions of your model probably talk about some random variables and in the case of regression it's typically that the errors are assumed to be i.i.d. N(0, sigma).

So let's say you took another sample of the same size as your original. Would you expect to get the exact same slopes and intercept? Probably not because there is randomness involved. Imagine you do this process (sample and then estimate parameters) over and over. You'll end up with a lot of estimates for the slopes and intercepts. You might even imagine it makes sense to talk about the distribution of those estimates. Let's just talk about the intercept estimate to keep things simple. So if you visualized what your intercept estimate distribution looked like over many samples we would be getting at what is called the sampling distribution of the intercept. It's the variance of *that* distribution that the video is referring to. Now why do we care about this? Because typically we have questions about the *true* intercept (or more likely the *true* slope). By using the sampling distribution we can start to do some statistical tests on some hypothesis we may have about the true parameter (i.e. is the parameter equal to 0 or do we have evidence that it isn't equal to 0?).
Many thanks for the response, it is not only logical but also simple for a rookie to understand.