Comparing two dichotomous variables (non-paired)

P

PAUL!!

New Member
#1
#1
Dear reader,

I would like to compare two dichotomous variables:
1) pass/fail for a statistics test before statistics class (pre-test)
2) pass/fail for a statistics test after statistics class (post-test)

I only have data on pass/fail (so dichotomous and not able to retrieve interval measure; therefore not able to execute a dependent t-test)
As my fellow students filled in the test anonymously I am not able to pair/match results of the pre-test and post-test (so I am not able to execute a McNemar Test).

Does anyone know which statistical test would be appropriate?
I hope to hear from you!

Kind regards,
Paul
 
C

css

Member
#2
#2
Paul, can you explain what would be the aim of your intended analyses? In this regard, you mention that you cannot pair pre-post answers from the same subjects. If that's the case, I do not see how you could statistically compare (you cannot pair them but you cannot treat them as independent, either) . The only thing I can envision is, if you have subjects enough, to calculate bootstrap estimates of the 95% confidence intervals (CI)of the proportion of "pass" (or "fail") in each condition and make a subjective consideration based on the values of these proportions and their CIs
 
K

katxt

Active Member
#3
#3
Here's a not very powerful way of using McNemar's test. Lets say you have 50 students. In the pretest the P:F was 20:30 and the posttest P:F was 40:10. You can make up a 2 way table with the marginal totals. Put n into cell a. Fill in the rest.
1621113944217.png
McNemar now says that under the null hypothesis (no real difference between pre and post, or differences are due to random chance) that X = (b - c)^2/(b + c) is distributed Chi squared 1 df. or in other words X should be less than 3.84.
In this case X = ((40-n)-(20-n))^2/(40-n + 20-n) = 20^2/(60 - 2n)
Minimum X happens when 60 - 2n is a maximum which is when n is a minimum. All cells are non negative so the smallest n can be is 10 making d = 0. So minimum X is 20^2/40 = 10 which is greater than 3.84, so there is a significant difference (no matter what n is.)
In short, set up the marginal totals, put either a or d = 0 (use the one which makes all cells non negative) and use McNemar.
Not powerful, but at least valid.
 
Karabiner

Karabiner

TS Contributor
#4
#4
PAUL!! said:
Dear reader,

I would like to compare two dichotomous variables:
1) pass/fail for a statistics test before statistics class (pre-test)
2) pass/fail for a statistics test after statistics class (post-test)
Click to expand...
What were the pass/fail frquencies at both time points?

With kind regards

Karabiner
 
K

katxt

Active Member
#5
#5
A similar calculation as above will give you the largest possible value of McNemar's X and could show that the difference is not significant for any value of n.
If by chance the minimum and maximum possible values of X straddle 3.84, then a Monte Carlo simulation would give the probability of the true X exceeding 3.84.
kat
 
You must log in or register to reply here.