matched pre-post survey analysis help

#1
I have matched pre-and post test data that evaluated our workshop participants using Likert-type items. Meaning, the same people were given the same questions before and after a workshop, and the questions they answered used the Likert format (not confident, somewhat confident, confident, very confident) but we did not sum up the items to make it an official Likert scale.

I did use a t-test to test for the means of the responses to the items and found a significant difference, though I know a t-test may not be considered robust for Likert-type questionnaire items (the literature seems divided on that). I could have used a Wilcoxon signed rank test (which btw also showed a significant difference), however I didn't have any evidence to believe the data were not normally distributed.

For presenting the data, I was recently reading about using the McNemar's test, since I dichotomized the answers for both pre and post test into a "confident" group and "not confident" group.

Would you be able to assist me in recommending that I use the McNemar's, stick with the t-test (despite it being controversial), or a different test more robust I'm not thinking of?

N=40.

Thank you!!
 

Masteras

TS Contributor
#2
Definetely no t-test and Wilcoxon because your data are not numerical, i.e. not numbers, but preferences.
McNemar's test is not appropriate because it is designed for paired dichotomous data.
You need a paired data test for ordinal data.
Stuart-Maxwell marginal homogeneity test for ordinal data. In R there is the package coin that does it. In Spss i do not think it exists.
 
#3
Well, the data are entered into my dataset are numbered accordingly=no confidence 1, low confidence 2, somewhat confident 3, very confident 4.
Interestingly, I consulted 4 other biostatisticians, and they all felt the Wilcoxon was best. So I appreciate you giving me more to think about!
With regards to McNemar's, I did in fact dichotomize the data into "more confident" (upper 2 groups) and "less confident" (lower 2 groups), so in that sense McNemar's test would use dichotomized data.
Thank you for your reply!
 

Masteras

TS Contributor
#4
Look, you agree that with dichotomised dependent data you do McNemar, right?
Wilcoxon or the t-test (Wilcoxon's parametric vesrion) are designed for numbers, 1, 2,3, 0.5, 0.6, -1, -45 and so on.
Do you have numbers? no confidence, low confidence etc. What is average of no confidence and somewhat confidence?
If you answer this then you can do t-test or Wilcoxon.