# Thread: Variable change from significant to not significant

1. ## Variable change from significant to not significant

I run a multiple regression with 4 main drivers and some control variables.

Then I had to run another regression but with six main drivers (of which 4 the same as the previous regression) and the same control variables.

Because I had to inlcude 2 more variables, I had to reduce my sample by 4.

When I compare the two outputs, one variable that was significant in the first regression has become not significant. Even when I omit the other 2 variables in the 2nd regression the variable that was significant in the 1st regression stays not significant.
Is it because in the 2nd regression my sample is slightly smaller?

2. ## Re: Variable change from significant to not significant

You could tell us how large your sample size was, and what was actually happening. "Significant" is not very informative (could mean p=0.049 or p=0.00000000001), same with "not significant" (p=0.05? p=0.98?). You could describe the regression weights, standard errors, and precise p values for the variables in question.

With kind regards

Karabiner

3. ## The Following User Says Thank You to Karabiner For This Useful Post:

Jazz3 (04-10-2017)

4. ## Re: Variable change from significant to not significant

also what happened to the beta coefficients?

5. ## The Following User Says Thank You to hlsmith For This Useful Post:

Jazz3 (04-10-2017)

6. ## Re: Variable change from significant to not significant

In the 1st regression, I have 4 main drivers and a total N of 348, I run the regression on robust because hettest was significant

Var /Coefficient/ robust std. error/ t / P value
X1/ .902 .1516 5.95 0.000
X2/ -.521 .1023 -5.10 0.000
X3/ .221 .1015 2.19 0.030
X4/ .037 .0823 0.45 0.650

In the 2nd regression I have six main drivers (of which 4 are from the previous regression) and a total N of 324, also run the regression on robust. X3 is the variable that becomes not significant.

Variable / coefficient / robust std. error/ t / P value
X1 .847 .1382 6.13 0.000
X2 -.5214 .0973 -5.36 0.000
X3 .0333 .1060 0.31 0.753
X4 -.081 .0763 -1.07 0.287
X5 .0088 .0858 0.10 0.918
X6 -.360 .1107 -3.25 0.001

The only difference in the sample between the 1st and the 2nd regression is that the sample is reduced by 4 (= dependent variable) so that I could include the other 2 variables in the 2nd regression. Because its over a time period of six years, I guess the total sample reduces then by 24

7. ## Re: Variable change from significant to not significant

Looks very much like in the 2nd model, X6 explains the variance previously
explained by X2.
Couldn't quite follow you with regard to number of missing cases (4? 24?).
What suprises me:
Even when I omit the other 2 variables in the 2nd regression the variable that was significant in the 1st regression stays not significant.
So what were the coefficients in that analysis (the same analysis as the first, but
with a reduced sample?).

Because its over a time period of six years
Does that mean that you followed-up the same subjects (n=58?) over
6 time periods? If yes, then your analysis is incorrrect if you treat subjects
measured repeatedly as if they were independent cases.

With kind regards

Krabiner

8. ## Re: Variable change from significant to not significant

Pay heed to Karabiners questions on your data structure.

Just for giggles, why don't you use model 1 on model 2's subsample and see what X3 looks like with and without the other new variables. You should also look at the collinearity between variables and since X3 decreases so much is there potential confounding or mediation between variables. You should draw out how you think the covariates influence each other and the dependent variable.

9. ## Re: Variable change from significant to not significant

Originally Posted by Karabiner
Looks very much like in the 2nd model, X6 explains the variance previously
explained by X2.
Couldn't quite follow you with regard to number of missing cases (4? 24?).
What suprises me:

So what were the coefficients in that analysis (the same analysis as the first, but
with a reduced sample?).

Does that mean that you followed-up the same subjects (n=58?) over
6 time periods? If yes, then your analysis is incorrrect if you treat subjects
measured repeatedly as if they were independent cases.

With kind regards

Krabiner

In the 2nd regression with the reduced sample

coeff std. error t p value
X1| .9149 .1452 6.30 0.000
X2| -.4557 .0970 -4.70 0.000
X3| .1360 .1022 1.33 0.184
X4| -.0579 .0807 -0.72 0.474

thus not the same as in the first regression

What I mean with the sample is that, I have 58 categories with their market shares over a six year period.
So my data is listed as followed:
column 1 column2 column3
year 1 category a share category a year 1
year 1 category ab share category ab year 1
year 1 category b share category b year 1
etc etc etc
year 2 category a
year 2 category ab
etc.
year 3 category a
year 3 category ab

Column 3 is my dependent variable

The 2nd regression has 4 categories less (Because I had to include the other two variables, and since for the 4 categories the variables weren't available I had to exclude them from the analysis) and thereby thus a total of 24 less observations than the first regression

I checked for collinearity and didn't find anything above .6

10. ## Re: Variable change from significant to not significant

You have small change of the coefficient for variable X3 between model 1 and model 3. Seemingly, the missingness of 24 values caused this difference, which in turn changed the p-value.

11. ## The Following User Says Thank You to Karabiner For This Useful Post:

Jazz3 (04-23-2017)

12. ## Re: Variable change from significant to not significant

Originally Posted by Karabiner
You have small change of the coefficient for variable X3 between model 1 and model 3. Seemingly, the missingness of 24 values caused this difference, which in turn changed the p-value.
I run the original model (where X3 was significant) on the sample of the 2nd model (where X3 is not significant) and X3 is significant.

I take this back:
"Even when I omit the other 2 variables in the 2nd regression the variable that was significant in the 1st regression stays not significant."

I overlooked: when I omit the 2 variables in the 2nd regression, X3 is significant.
Thus I included X5 and X3 was still significant, when I added X6, X3 became insignificant. Thus I checked for correlation

But between X3 and X6 the correlation is "only" -0.5 (X3 and X1 correlate on -0.7, but that's not the issue here?)

 Tweet

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts