Why is there n-1 degrees of freedom and not n-2 (student distribution)

#1
When we want to compute a confidence interval for the mean of a sample of n persons, it is always explained that the number of degrees of freedom used should be equal to the sample size minus the number of estimated parameters.
But, in my opinion, two parameters are estimated: the mean is estimated by the sample mean and the variance is estimated by the sample variance (corrected). Then why shouldn't we use the t-distribution with n-2 degrees of freedom?
 

Dason

Ambassador to the humans
#4
The estimate of the sample mean is required before estimating the sample variance. So the number of parameters you need to estimate before you actually have enough information to estimate the variance is the number that is important for the degrees of freedom for the variance.

Go ahead and work it out - try and estimate the variance without first having some estimate of the mean (you won't be able to).
 
#5
Yes I understand that the variance has n-1 degrees of freedom because of the estimation of the sample mean.
BUT the t-distribution uses the estimate of the sample variance, which should remove ANOTHER degree of freedom?

I hope my misunderstanding is clear enough?
 

Dason

Ambassador to the humans
#6
To provide a counter point to your assertion that what you're describing should remove another degree of freedom:

why?
 
#7
Hi Rafaelle ,

I will try to say in other words what Dason said.

The refection of the DF is Not for any parameter you estimate BUT because you estimate the Variance based on the estimation of the Mean, so there is a connection between the mean and the variance and here you loose the degree of freedom.

For example in z test you only estimate the mean, so you don't loose any degree of freedom.
 
#8
Hi,

So if I understand well, this statement is wrong?
"Why do we have n-1 degrees of freedom? The reason is because to calculate the test statistic, we have used the sample mean as an estimator for the population mean µ"
 
#10
Hi Dason and Rafaelle.

My common sense says n-2 also I understand it is not correct as all know it is n-1...
if you have 5 values: x1,x2,x3,x4,x5
if you know the estimation of mean - average(x) and estimation of variance - sample var(x) and you know x1 x2 and x3 you can calculate the x4 and x5

I found the following explanation not sure if correct... He wrote:
"Also, note in a hypothesis test involving a 1 sample t-distribution the only known is the mean mu. The standard deviation/variance is estimated from the sample size so is not known. If you do know the standard deviation of the population then you should be using a Z-test not a T-test."

https://www.quora.com/Why-is-the-de...mean-and-the-standard-variance-are-both-given
 
#11
I may be able to add some clarity if you are familiar with MLE (maximum likelihood estimator) methods. In the process of finding the MLEs of a normal distribution (which the CLT [central limit theorem] tells us a sampling distribution converges to) we are looking to find the mean and variance. When finding the MLEs simultaneously you will find that the two parameters are independent (this can be verified with the fisher information matrix and its inverse the correlation matrix).

In short, the mean of a normal distribution does not rely on the variance and the variance does not rely on the mean, therefor when conducting inferences you only lose 1 degree of freedom. Also, the t-distribution is by definition has a mean of zero, therefor the only parameter unknown is the variance, again resulting in the 1 less degree of freedom.

If you still aren't convinced, write some R code and run a few million iterations. I do this sometimes, because I am skeptical at times too.
 
#12
My common sense says n-2 also I understand it is not correct as all know it is n-1...
if you have 5 values: x1,x2,x3,x4,x5
if you know the estimation of mean - average(x) and estimation of variance - sample var(x) and you know x1 x2 and x3 you can calculate the x4 and x5
That is exactly my "problem" and I feel like I'm not convinced why it is n-1
 
#13
I found the following explanation not sure if correct... He wrote:
"Also, note in a hypothesis test involving a 1 sample t-distribution the only known is the mean mu. The standard deviation/variance is estimated from the sample size so is not known. If you do know the standard deviation of the population then you should be using a Z-test not a T-test."

https://www.quora.com/Why-is-the-de...mean-and-the-standard-variance-are-both-given
I think this is different; my case is when the standard deviation is estimated and therefore not known. Of course when the standard deviation is known, a Z-test should be used.
 
#14
In short, the mean of a normal distribution does not rely on the variance and the variance does not rely on the mean, therefor when conducting inferences you only lose 1 degree of freedom.
Because they are independent, you should lose 2 degrees of freedom (two independent pieces of information)? If they were dependant, I would understand n-1.

Sorry if I really don't understand
 

Dason

Ambassador to the humans
#15
I think this is different; my case is when the standard deviation is estimated and therefore not known. Of course when the standard deviation is known, a Z-test should be used.
So you're fine with there being no mention of degrees of freedom in a Z test even though we estimate the mean in the process?
 
#16
Rafaelle, you are not having to guess the variance at the same time as guessing the mean.

Think of the data points creating the hyper-plane of n dimension covering all possibilities of data. Then the mean and variance in this case can have all possibilities in a 1 dimensional vector in this case (each having their own). So the resulting hyper plane is of the dimension of possibilities minus the maximum dimension of parameters.

If you have a distribution like a gamma function where the two a parameters are dependent, then the possibilities of the parameters is a 2 dimensional plane. Then the resulting degrees of freedom as n - 2.

I do encourage you to write some code and see for yourself. Create a random sample of normal data with known mean and variance. Then conduct the interval for the mean using degrees freedom of n-1 and n-2. Repeat this a couple thousand times and compare the reliability of the intervals against the intervals confidence. Doing a 95% CI you will see that n-1 will capture the mean 95% of the time, but n-2 will capture it more than 95% of the time.
 

Dason

Ambassador to the humans
#18
That's actually a good thing that you're not sure anymore.

The degrees of freedom really are associated with the variance estimate. In the case of a z-test we *know* the variance so we don't need to estimate anything before we can get what we use for the variance. In the case of a t-test we need to estimate the mean before we can even estimate the variance since it is required for the variance calculation. That's why we lose one degree of freedom - we needed to estimate one parameter before we could estimate the variance. In a simple linear regression we need to estimate the intercept and the slope before we can get a prediction for each point which is required before we estimate the variance. So in that case we lose 2 degrees of freedom because we estimated 2 parameters before we could estimate the variance.
 
#19
Rafaelle, you are not having to guess the variance at the same time as guessing the mean.

Think of the data points creating the hyper-plane of n dimension covering all possibilities of data. Then the mean and variance in this case can have all possibilities in a 1 dimensional vector in this case (each having their own). So the resulting hyper plane is of the dimension of possibilities minus the maximum dimension of parameters.

If you have a distribution like a gamma function where the two a parameters are dependent, then the possibilities of the parameters is a 2 dimensional plane. Then the resulting degrees of freedom as n - 2.

I do encourage you to write some code and see for yourself. Create a random sample of normal data with known mean and variance. Then conduct the interval for the mean using degrees freedom of n-1 and n-2. Repeat this a couple thousand times and compare the reliability of the intervals against the intervals confidence. Doing a 95% CI you will see that n-1 will capture the mean 95% of the time, but n-2 will capture it more than 95% of the time.
Aren't the mean and the variance dependent?
 
#20
That's actually a good thing that you're not sure anymore.

The degrees of freedom really are associated with the variance estimate. In the case of a z-test we *know* the variance so we don't need to estimate anything before we can get what we use for the variance. In the case of a t-test we need to estimate the mean before we can even estimate the variance since it is required for the variance calculation. That's why we lose one degree of freedom - we needed to estimate one parameter before we could estimate the variance. In a simple linear regression we need to estimate the intercept and the slope before we can get a prediction for each point which is required before we estimate the variance. So in that case we lose 2 degrees of freedom because we estimated 2 parameters before we could estimate the variance.
That makes sense to me. I think I'm beginning to understand, thank you very much for your answers.

A quick question though: why are the degrees of freedom associated with the variance estimate? Is it linked to the definition of a "degree of freedom"?
Maybe you can use this example to explain me the definition of a degree of freedom (more accurately, where I misunderstand the definition of a degree of freedom):
if you have 5 values: x1,x2,x3,x4,x5
if you know the estimation of mean - average(x) and estimation of variance - sample var(x) and you know x1 x2 and x3 you can calculate the x4 and x5