t-test or ANOVA? urgent

[K psychology]

Hi,

I am looking for advice about which stat test to use in my data analysis for our psychology class. We collected data on reasoning tests scores.

I have two groups of participants 1) non-bilingual 2) bilingual and dependent variable in the form of reasoning scores in 1) spatial reasoning test and 2) verbal reasoning test. The verbal and spatial tests add up to one reasoning test that was applied at one sitting.

I would want to find out whether there is a difference in the performance in spatial and verbal test between non-bilingual and bilingual participants.

I wanted to use independent sample t-test and compare scores between non-bilingual and bilingual in the spatial domain (to produce t value) and in the verbal domain (to produce another t value). I got non-significant p values at both those tests.

However I have realised that his may lead to multiple comparisons! Should I instead use ANOVA and if so which type? Or should I jsue the t-test but adjust the desired p-value for multiple comparisons?

Thank you in advance, it is very urgent!

 

[obh]

Hi K,

I don't understand.
So you compare only two groups? So why multiple comparisons?
ANOVA with two groups and t-test with equal variances are identical tests (the same p-value)

 

[K psychology]

Hi Obh,

Thanks a lot for your swift reply.

Yes I am only testing two groups. However each group would be tested twice (i.e. one t-test for spatial score and second t-test for verbal score).

 

[K psychology]

My question comes from the comment made by my teacher.

Basically, we carried out one big intelligence test that consisted of 1) spatial and 2) verbal subparts.

However, WITHIN both spatial and verbal subparts there were two types of questions 1) categorical and 2) analogical. Therefore we end up having scores for 1) spatial categorical scores 2) spatial analogical scores 3) verbal categorical scores and 4)verbal analogical scores. We would have those 4 subgroups for both non-bilingual and bilingual people.

I also wanted to carry out another analysis: Is there a difference in in the performance in categorical (spatial categorical + verbal categorical scores) versus analogical (spatial analogical +verbal analogical scores) between bilingual and non-bilingual people. I wanted to use independent t-test here. However my teacher said this would be inappropriate.

I thought that comparing spatial vs analogical scores for 2 groups is analogical to comparing spatial and verbal for 2 groups (which I described in this query). Therefore I do not undertand why my teacher would say t test is inappropriate and it make me think that test is also inappropriate for my question described in the first post.

Thank you

 

[obh]

Hi K,

You should always check that your case meets the assumption of the test you use.

Do you want to check each question separately? or the average of the questions?
What are the combinations that you want to check?

Can you please show a simple demo example?

If you run two separate t-tests than the allowed significant level is bigger for the overall comparisons.
If for example, you use α=0.05
α′=1−(1−α)^n=1-(1-0.05)^2= 0.0975
So the type 1 error from the combination of the 2 tests is 0.0975, bigger than the 0.05 for a single comparison.

To correct this you can change the significant level using the Bonferroni correction.
Or better run the Tukey HSD test (or actually Tukey-Kramer test for uneven groups)

 

[K psychology]

Hi Obh,

I really appreciate your time.

I want to check if there is a difference in spatial and verbal scores depending on if someone is bilingual or non-bilingual.

I was actually thinking that repeated measures 2 way ANOVA could be the best solution. There would be within subjects factor (spatial vs verbal test) and between subject factor (blingual or non-bilingual).

I will then see whether there are main effects of bilinguality, test domain (spatial or verbal) and interaction.

HOWEVER as you advised I just tested the assumption. I cannot carry out the ANOVA because of Box M test is significant (so there is violated assumption of equality of the covariance).

 

[obh]

Hi K,

If I understand you correctly... (I hope), it is not repeated ANOVA/paired-t since you don't check the DV twice, but you check 2 different independent DVs

Two way ANOVA is one DV with two IVs.(the factors)

Two way ANOVA is like Y=f(x1,x2) while as I understand your case is Y1=f(x1) , Y2=f(x1)

Y1-spatial score
Y2- verbal score
x1 - bilingual /non-bilingual

PS Interesting research
I'm pretty sure that the verbal score will be better for the bilinguals.

 

[K psychology]

Dear Obh,

Thank you so much for your involvement!

I have actually decided to use my dataset to investigate something else - whether bilingualism affects performance at analogical and categorical reasoning.

Specifically, I have a set of participants who were either bilingual or non-bilingual. Each of those participants solved a reasoning test which consisted of two types of questions: analogical type of questions (e.g. A is to B as C is to ?) and categorical type of questions (the odd one out).

I have standarised the scores to control for different number of questions. I have also used square root data transformation to meet the assumption of normality (which was not met without the trasnformation).

I plan to use MIXED DESIGN ANOVA, with bilingualism as between-subject factor (two levels: bilingual and non-bilingual) and the type of reasoning questions as within subject factor (two levels: categorical and analogical).

Could you please confirm whether that would be appropriate?

 

[Karabiner]

Normality is not needed for a dependent variable. Normality of the residuals might be of relevance, but only if sample size is small. So don’t transform data if there is no substantial reason for it; how will you ever interpret results like “there was a statistical significant effect of group on the square root of categorical thinking“?

The research question is not quite clear to me. Maybe you should just perform 2 t-tests, with Bonferroni correction of the p-values, if you are afraid of multiple testing issues.

With kind regards

Karabiner

 

[obh]

Dear Obh,

Thank you so much for your involvement!

I have actually decided to use my dataset to investigate something else - whether bilingualism affects performance at analogical and categorical reasoning.

Specifically, I have a set of participants who were either bilingual or non-bilingual. Each of those participants solved a reasoning test which consisted of two types of questions: analogical type of questions (e.g. A is to B as C is to ?) and categorical type of questions (the odd one out).

I have standarised the scores to control for different number of questions. I have also used square root data transformation to meet the assumption of normality (which was not met without the trasnformation).

I plan to use MIXED DESIGN ANOVA, with bilingualism as between-subject factor (two levels: bilingual and non-bilingual) and the type of reasoning questions as within subject factor (two levels: categorical and analogical).

Could you please confirm whether that would be appropriate?

I was trying to lead you to the same direction as @Karabiner.
I also think you should go for 2 t-tests with Bonferroni correction (actually better use the Sidak correction with is more accurate:
α=1−n√(1−α′)