MANOVA vs ANOVA

WeeG

TS Contributor
#1
Hi guys,

I have a small question. I ran a MANOVA model, with 4 DV's and 1 IV with 2 categories. I got a non significant result, with P. Value of 0.189 (for every criteria). The output (SPSS) also gives the 4 single ANOVA's, and one of them WAS significant with P Value of 0.03.

What kind of conclusion can I give in such a situation ?
Can I say that there is no relation between the IV and the group of DV's, but there is a relation between the IV and DV3 ?
 

Dragan

Super Moderator
#2
Hi guys,

I have a small question. I ran a MANOVA model, with 4 DV's and 1 IV with 2 categories. I got a non significant result, with P. Value of 0.189 (for every criteria). The output (SPSS) also gives the 4 single ANOVA's, and one of them WAS significant with P Value of 0.03.

What kind of conclusion can I give in such a situation ?
Can I say that there is no relation between the IV and the group of DV's, but there is a relation between the IV and DV3 ?
How are you controlling for Family-Wise error across your 4 single ANOVA's??

A Dunn-Sidak approach would indicate a critical value p-value of .0127 per test (i.e. instead of 0.05 per test).
 

WeeG

TS Contributor
#3
I am not sure I understand your question. I ran 4 ANOVA's (one way anova), and 1 of them was significant. but the MANOVA is not, what would be your conclution ?

some more details:

Pillai's Trace: Value=0.165 F=1.63 Hypothesis df=4 error df=33 P=0.189

Test's of between subject effects (ANOVA):
3rd DV: type III some of squares=3.003 df=1 mean square=3.03 F=5.117 P=0.03
 

Dragan

Super Moderator
#4
I am not sure I understand your question. I ran 4 ANOVA's (one way anova), and 1 of them was significant.

Okay, let me try this.

Are you using a Type I error rate of 0.05 for each of the 4 ANOVAs as a basis to make a decision as to whether or not any given ANOVA is significant?

For example, are you suggesting that because p=0.03 < 0.05 for the 3rd DV, then this ANOVA is a significant result?

If this is the case, then, in my view, is not appropriate.

In short, you cannot control for Family Wise error by performing a series of ANOVAs.
 

WeeG

TS Contributor
#5
OK, you say that I can't say that a series of DV's are significant if one ANOVA is significant, I agree on that. but can I say something like: the family containing 4 DV's is NOT significant, but the individual DV number 3 is significant. I mean, if I didn't have the other DV's I would have ran ANOVA, right ? and then I would have got a significant result, so should I take notice to that or just ignore the significant ANOVA ?
and yes, I was using a error rate of 0.05.
 

Dragan

Super Moderator
#6
OK, you say that I can't say that a series of DV's are significant if one ANOVA is significant, I agree on that. but can I say something like: the family containing 4 DV's is NOT significant, but the individual DV number 3 is significant. I mean, if I didn't have the other DV's I would have ran ANOVA, right ? and then I would have got a significant result, so should I take notice to that or just ignore the significant ANOVA ?
and yes, I was using a error rate of 0.05.

What I am saying is that your overall (or Family) Type I error rate is not 0.05 for each of the 4 tests because your conducting 4 separate ANOVAs. More specifically, if we assume the 4 tests are independent then your Type I error rate is

FamilyError = 1 - (1 - .05)^4 = 0.185.

In other words, in the family of 4 tests your potential to make a Type I error is .185 and not .05.

This is why I suggested the Dunn-Sidak downward adjustment on the p-value from 0.05 to 0.0127 for each of the 4 ANOVAs.

Thus, because p=0.03 > 0.0127 we would conclude that it is not a significant result for the 3rd DV.
 

WeeG

TS Contributor
#7
now I understand what you mean, right, so now I am much more relaxed about the results, they are simply no significant, meaning no relation. I checked the equally of the error variance using Levene's test, and I filtered outliers that I found using Mahalanobis D^2. I really hope that the linearity is OK, I didn't know how to check it. The normality is also an issue due to small sample size
 

Lazar

Phineas Packard
#8
Coming back to your original question, typically if the MANOVA results are not significant you do not look at the ANOVA results (full stop). As pointed out, when you run multiple ANOVAs you increase your probability of finding a significant result that is due to chance alone (ie. Type I error). The reason you use MANOVA is that it acts as a protection of this familywise error. However, it only protects against Type I error if you don't go poking around the univariate results if the MANOVA is not significant.

If your MANOVA was significant you would be justified at looking at the univariate results at .05 rather than doing any sort of correction.