# Thread: The joint normality of the two estimates for linear regression model

1. ## The joint normality of the two estimates for linear regression model

I have a different opinion with my colleagues on this question. Could someone help us to clarify this issue?

where , , and , is iid. The estimate of is , . Now we suggest another linear model,

where and are same as above and .
For this linear model, we have the estimate of is . We know that and are normal respectively. Now the question is that whether the joint distribution of and is normal as well?

2. ## Re: The joint normality of the two estimates for linear regression model

Assumptions and well-known facts:

1. are i.i.d. as

2. are constants.

Therefore and they are independent

Most importantly, jointly follows a multivariate normal distribution as they are independent.

As is an affine transformation of the multivariate normal vector, it has a normal distribution.

The question now comes:

What is ? Is it defined to be like which has non-zero mean?

The variables "in the second model" is another independent sample or what?

Are and independent?

4. ## Re: The joint normality of the two estimates for linear regression model

BGMs concerns still hold though. You can't just say that Y and X2 are the same as above and then claim that is normally distributed with a mean of 0.

5. ## Re: The joint normality of the two estimates for linear regression model

Could we just ignore the contradiction between these two models and just talk about the normality of these two estimates? Actually, this happens in practice. The first model is regarded as a full model and the second model is misspecified (one ore more covariates are omitted from the model). The is because, sometimes, you fail to observe all the covariates in the model.

6. ## Re: The joint normality of the two estimates for linear regression model

So let say the first model is the "true" model. Then as what you written,

As long as is a multivariate normal vector, and those are constants, then the above estimators are just affine transformation of and thus again they are jointly distributed as a multivariate normal distribution. Of course you have to ensure the transformation has full rank.

7. ## Re: The joint normality of the two estimates for linear regression model

Originally Posted by BGM
So let say the first model is the "true" model. Then as what you written,

As long as is a multivariate normal vector, and those are constants, then the above estimators are just affine transformation of and thus again they are jointly distributed as a multivariate normal distribution. Of course you have to ensure the transformation has full rank.
We can't say the transformation has full rank, because

[LaTeX Error: String is too long (806, limit 400)]

8. ## Re: The joint normality of the two estimates for linear regression model

Originally Posted by BGM
So let say the first model is the "true" model. Then as what you written,

As long as is a multivariate normal vector, and those are constants, then the above estimators are just affine transformation of and thus again they are jointly distributed as a multivariate normal distribution. Of course you have to ensure the transformation has full rank.
We can't say the transformation has full rank, because

We know that the is joint normal, because is independently normal and has full row rank. But, the has not full row rank, because the has full rank.
Does this mean that is not joint normal?

9. ## Re: The joint normality of the two estimates for linear regression model

I give you a simple example. Suppose is a multivariate normal random vector.

Next consider the random vector

where are constants.

So will you say that the joint distribution of this vector does not exist, when and are not linearly independent?

In other words, when the rank of the transformation matrix is 1, it is a degenerate one. When it has a full rank of 2, you still have a multivariate normal random vector.

10. ## Re: The joint normality of the two estimates for linear regression model

Originally Posted by BGM
I give you a simple example. Suppose is a multivariate normal random vector.

Next consider the random vector

where are constants.

So will you say that the joint distribution of this vector does not exist, when and are not linearly independent?

In other words, when the rank of the transformation matrix is 1, it is a degenerate one. When it has a full rank of 2, you still have a multivariate normal random vector.
This is the point. Can we say is bivariate normal, when and are linearly dependent?
For example, we take and . In this case, and are linearly dependent.
Since,

and . You mean [math] \begin{pmatrix}Z_1\\Z_2 \end{pmatrix} is normal?

, where is normal distribution function.
Can we say is a normal distribution?

11. ## Re: The joint normality of the two estimates for linear regression model

Originally Posted by BGM
I give you a simple example. Suppose is a multivariate normal random vector.

Next consider the random vector

where are constants.

So will you say that the joint distribution of this vector does not exist, when and are not linearly independent?

In other words, when the rank of the transformation matrix is 1, it is a degenerate one. When it has a full rank of 2, you still have a multivariate normal random vector.
This is the point. Can we say is normal, when and are linearly dependent?
For example, , in this case, and are linearly dependent.
Since,

and . You mean is normal?

, where is normal distribution function of .
Can we say is a normal distribution?

12. ## Re: The joint normality of the two estimates for linear regression model

As I posted before, I think degenerate Multivariate normal distribution is a proper name to describe this. The nomenclature alone is not important - as long as you know the "dimension" of the resulting vector it should be fine.

Some will say a constant value has a degenerate distribution. So you may think that a constant and a random variable has a degenerate joint distribution, but that maybe odd to other people.

13. ## Re: The joint normality of the two estimates for linear regression model

Originally Posted by BGM
As I posted before, I think degenerate Multivariate normal distribution is a proper name to describe this. The nomenclature alone is not important - as long as you know the "dimension" of the resulting vector it should be fine.

Some will say a constant value has a degenerate distribution. So you may think that a constant and a random variable has a degenerate joint distribution, but that maybe odd to other people.
Thanks for your reply. But I am a little confused about this degenerate Multivariate normal distribution. Is it still a normal distribution or not? For me, I think it is not normal. Because the bivariate normal distribution can be expressed as

where
I don't think can be expressed like the above formula.

14. ## Re: The joint normality of the two estimates for linear regression model

Sorry I think we are either off topic or lost the focus.

Again I can give a even more simple example: Let be a univariate normal random variable. Consider the normal random vector

Actually you know the behavior of this random vector very well - if you consider it as the coordinates on the plane, then it represents a point normally distributed on the line . So you know how to simulate/generate it. You have all the information related to this random vector. You can describe it to the other people. You can even try to give a name like a "degenerate" multivariate normal. Yet you cannot write down a joint pdf for it (of course that is natural as it is degenerate. In multivariate normal case, you may think that is due to the variance-covariance is not invertible as it does not have a full rank) . Is it the only trouble? I do not think it bother your study about this object.

 Tweet

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts