Determining the Standard Deviation (vice Standard Error) of regression coefficients?

LKM

New Member
#1
This is a question I’ve been wondering about for a while, but perhaps it’s not possible or perhaps I just need a point in the right direction.

I constructed the attached mock data samples with four variables, A, B, C, D, each of which is either present (1) or not (0). The true value for each is 2, 4, 6, and 8, respectively, and each has a standard deviation of 0.5, 1, 3, and 10, respectively. I generated 1000 observations using the RAND() function (since copied and pasted as values) and the results of each observation is in column L of the Original Data worksheet. I then ran Excel’s regression analysis (constant set to zero) and came up with the regression results in the Regression Results worksheet.

This shows the standard error for each coefficient as all being relatively close (approx. 0.42), but looking at the data clearly the bulk of the deviation comes from the inclusion of variable D. In fact, variable D appears in slightly less than half of all trials (48%), but nearly all of the SSE comes from the trials where D is present (95%).

It would seem to me there would be some way to capture this.

So, figuring the SSE would be a function of the standard deviation for each coefficient, I then ran a regression on the SSE thinking the result would be the variance of each coefficient, but then I end up with negative numbers. I’ve tried other data sets and I almost always get similar results.

It seems to me if the standard deviation of each coefficient can be estimated, then a better measure of standard error for each coefficient can be calculated, but obviously this is not the case since it isn’t done this way.

Am I missing something or going about this the wrong way?

LKM
 

LKM

New Member
#2
Re: Determining the Standard Deviation (vice Standard Error) of regression coefficien

I hope everyone's having a good holiday season.

Just checking to see if there might be any thoughts on this problem.
 

hlsmith

Less is more. Stay pure. Stay poor.
#3
Re: Determining the Standard Deviation (vice Standard Error) of regression coefficien

Perhaps if you provide a more concise/rephasing you will get more feedback.

If I followed,

you are saying that the SE of the beta coefficients are all about the same but the standard errors for the model reveal SD that show greater discrepancies between the independent terms. And you are wondering why we don't review SD for these models, correct?
 

LKM

New Member
#4
Re: Determining the Standard Deviation (vice Standard Error) of regression coefficien

Yes, I am wondering if there is a method to look at the SD for each beta coefficient and if so why isn't it a standard part of what is reviewed. The method. I tried, running a regression on the variance itself, doesn't seem to work. If it can be calculated, it would seem more accurate than the SE as calculated by using the covariance matrix.
 

LKM

New Member
#5
Re: Determining the Standard Deviation (vice Standard Error) of regression coefficien

Just wanted to bump this one more time - Anyone know of a method to determine the standard deviation of each beta coefficient and why this might not be preferred to using standard error?
 

Dason

Ambassador to the humans
#6
Re: Determining the Standard Deviation (vice Standard Error) of regression coefficien

It's not clear to me how your data is actual being generated. But it sounds to me like your situation isn't the typical regression situation.
 

LKM

New Member
#7
Re: Determining the Standard Deviation (vice Standard Error) of regression coefficien

Thanks for the reply, Dason.

I don't have a specific application at the moment to present, but imagine three variables, A = 2 and B = 3 with zero standard deviation while C = 7 with a standard deviation of 15. Anytime I have A + B then the answer is always 5 with no error, but whenever I have A + C the answer is 9 with a standard deviation of 15 and B + C = 10 with a standard deviation of 15.

It's easy to spot in the example I just gave, but if I had the coefficients in a matrix, such as:

1 1 0
1 0 1
0 1 1

And if I had 10 beta coefficients and each had a standard deviation above zero, then how can I find out what the SD is for each beta coefficient?