Short question; when do I use Welch's t-test? I don't quite get the "unequal variance"-thing. What does it mean that there is unequal variance and in what?
One of the underlying assumption of a t-test is the equality of variances between the two groups. If this assumption is violated then your test won't be valid and results won't be reliable.
Example: Suppose measurements were recorded on two arms following treatment.
Code:
## fake data
x<-c(9, 7, 6, 7, 5, 2) # control arm
y<-c(22, 5, 9, 12, 17) # treatment arm
Before performing a two-sample t-test, test if the variances are equal or not
Code:
> var.test(x,y)
F test to compare two variances
data: x and y
F = 0.1258, num df = 5, denom df = 4, p-value = 0.04305
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
0.01343831 0.92971147
sample estimates:
ratio of variances
0.1258427
The F-test is significant which means that the variances are not equal between two groups. Hence, performing a regular two-sample t-test won't be valid.
What happens if you ignore the underlying assumption of equality of variance? Here are the results:
Code:
## assuming equal variance
> t.test(x,y, var.equal=TRUE)
Two Sample t-test
data: x and y
t = -2.4163, df = 9, p-value = 0.03885
## The reality: Use Welch-two sample t-test as the varainces are not equal
t.test(x,y, var.equal=FALSE)
Welch Two Sample t-test
data: x and y
t = -2.2323, df = 4.84, p-value = 0.07772
Had you ignored the fact that your variances between the group are not equal, you will be performing a regular two-sample t-test and happy to find that there was difference in mean between two groups (p=0.03).
However, if you were careful in detecting the inequality of variance, then you'll use Welch and the evidence of mean difference between the two arms is not as bigger as before (p=0.07).
PS: Although, in my eyes, P=0.03 and P=0.07 aren't that different (I don't follow the 0.05 level religious. Instead I take both of them as significant), I hope this example drives home the message.
HTH