Interpreting confidence bands on interaction terms

[Kaga84]

Dear all,
I have a model with three variables a dependent (dep), a binary independent variable (bin) and a contin. independent variable (con). I am basically interested in whether the effect of con on dep is heterogeneous with respect to bin.

I regress:
dep=alpha0+alpha1*con+alpha2*bin+alpha3*con X bin + u

alpha1 is positive and significant, alpha3 is negative and signifcant. So I conclude that the effect is heterogeneous.

I then look at size of the effect of con on dep for the two groups:

1. For the baseline group (bin=0) the coefficient is: alpha1 and the standard error is the standard error on alpha1.
   
2. For the alternative group (bin=1) the effect should be: alpha1+alpha3 and the standard error is their combined standard error se(alpha1+alpha3).

My problem is now: when i compute confidence bands for the effects for each groups, the bands clearly overlap. So normally I would say that they are not statistically different. But my interaction term is very significant. What is wrong in my interpretation?

Many thanks in advance!
Kaga

[GretaGarbo]

Maybe this is analogous to the case when you compare two means. That’s an easier situation and maybe easier to compare.

Assume that you compare two means with known and equal population standard deviation (sigma) and both with equal number of observations (n).

Then you could draw a confidence interval like +/- 1.96*sigma/sqrt(n) for each of the two means. For each of the two intervals to not intersect the difference between the two means need to be 2*1.96*sigma/sqrt(n). Because you would go up wise from one mean and down wise for the other mean. But this in not the correct way to test if it is a statistical difference between the two means.

The standard way to test, or calculate a confidence interval for the difference of the two means is:

1.96*sqrt(sigma^2/n + sigma^2/n) = 1.96*sqrt(2*sigma^2/n)= 1.96*sqrt(2)*sigma/sqrt(n) and the last distance is less than the distance that was mentioned above.

This can be very pedagogically difficult. Suppose that you have two bars (for the means) in a bar chart. And that you also have small “error lines” up and down from the top of the bars. Then, even if the error lines (the confidence intervals) overlap a little bit, the two means can still be statistically significant. If the readers draw by hand a little bar chart I think this will be clearer.


I think the situation will be more clear if Kaga84 make up an example with two factors each with two levels and with an interaction in the model.

[Kaga84]

Many thanks!

Yes, I think it is the same issue. I found a similar explanation here.

THANK YOU