Help Understanding Difference Between Statistical Methods

#1
Have a data set of means for the perceived intensity (2) set of (2) samples. The samples were shown in pairs; however, all means were generated at the same time. I would like to understand if the perceived intensity (mean) of the samples within each pair are different from each other. Intensity scores were generated on a scale from 0 - 10.

From what I have read a Two sample (Paired) T-test would be the appropriate statistical analysis to conduct on EACH set of samples.

I also attempted and ANOVA with Tukey's post hoc of all (4) samples as all data was generated in the same session (test may not be as appropriate to run as the T-Test but was curious).

The T-Test resulted in a significant difference in mean scores (despite there being a small difference between the numerical values (for the scale I was using). The ANOVA did not identify the mean scores of these samples as statistically significant.

What is the reason these two statistical analysis approaches make different statistical conclusions? What about each analysis does this?

I know the T-Test looks at distance between the individual values from each respondent while the ANOVA looks at various points of variance in your data set to make a conclusion on if means are different.

Any help would be greatly appreciated to help be further understand the difference between these statistical approaches.

Thank you!!!
 

Karabiner

TS Contributor
#2
What is the topic of your study and what are the research questions? What does your study design look like? Which variables did you measure, and how (in detail)? How large is your sample size?

With kind regards

Karabiner
 
#3
Karabiner

The topic of my study was regarding fragrance intensity. My study design included having participants smell (2) sets of (2) samples. Samples within a set were balanced in presentation order and sets were balanced in presentation order.

I measured fragrance intensity only; however, also incorporate JUDGE as a factor in my ANOVA analysis.

My sample size was only 40 respondents.

Thanks so much in advance for any insights you can provide on this question.
 

obh

Active Member
#4
Hi,

I can't say I totally understand your question, so not sure if the following is relevant.

One way ANOVA with 2 groups (right tail) will provide the same p-value as Two-sample T-test Pool variance - 2 tails.
both assume 2 independent samples.

When you run the paired t-test the subjects are independent but the 2 samples are dependent as it uses the same subjects (in this example), therefore smaller standard deviation. so you won't get the sample p-value as 2 independent samples.
 
#6
@obh That makes sense. I was running a paired t-test vs. an ANOVA and was getting different P-values so that makes sense. What about each statistical analysis causes this? Also I used a 2 way ANOVA.

@Karabiner why would you select a repeated measures ANOVA?

Thank you both for your input on this!
 

obh

Active Member
#7
@obh That makes sense. I was running a paired t-test vs. an ANOVA and was getting different P-values so that makes sense. What about each statistical analysis causes this? Also I used a 2 way ANOVA.
When you use a"repeated measures" method like the paired t you have more control over the environment because you use the same subjects.
The repeated data is dependent data, you can't use 2 way ANOVA since you violate the independence of observations assumption.

@Karabiner. why would you select a repeated measures ANOVA?
Since you use repeated data (see above).