ANOVA One-way Vs Factorial ANOVA. Discordant results.

[nelson85]

Hi everybody,
I have to investigate the influence of many categorical factors (A,B,C) on a continuous response variable (Y). The procedure I'm following is:
1) Make three One-way ANOVA, every time with a different predicotor A, B, C)
2) Make all the three possible ANOVA two-way using the combination of two of the three factors (A with B, A with C and B with C)
3) Make e general linear model with all the three predictors (A with B with C)
If I look at the results I can observe differet outcome from the three analysis (point 1 ,2 and 3), for example: the one way ANOVA with A as a predictor says that there is no influence of A on the response Y, while the two way ANOVA says that A and the interaction A*B influence the response Y. This is just one of the several contrast that I found between the three ANOVA I had performed.
So, my question is: which of the three analysis (1,2,3) is more reliable? Which result should be considered?
Can I use the R square adjusted as a benchmark parameter?
Thank you in advance.

N.F.

[hlsmith]

Overall significance, r-squared, individual parameter significance are all informative. However, if you have significant interaction terms you do not want to miss them, since without them you may not be reporting the full story.

[nelson85]

Hi hlsmith,
first of all thank you for your answer. So, in your opinion I should go with the three way ANOVA considering the interaction and don't trust in the results of One and Two way ANOVA, shouldn't I? Do you think I have to perform a multicollinearity analysis on the predictor? Which one is the best for categorical predictor?
Thank you

[hlsmith]

Yes, controlling for multiple variables at once will help you understand interactions. Yes, if there is threat of collinearity you should test for it. Once you start better understanding the data another question that should come up is if you have a adequate sample size to support the saturated model?

[nelson85]

I'm working on historical data and unfortunately I don't have balanced design. In fact I'm paying particular attention on respecting the test assumptions.
I would like to ask you which is in your opinion the best way to check for collinearity with both qualitative and quantitative data.
Thank you!

[hlsmith]

For continuous data many people use Tolerance and Variance Inflation Factor (VIF). For categorical data (that can be ranked), some people recode it as intergers and run Tolerance and VIF.