Which Statistical Test?

#1
Hey all. I just performed a really nifty evaluation of a Zoo's educational program to determine the change in children's humane attitudes/behaviors and I'm having issues choosing a test. Here are some of the details:

Participants took before and after surveys. Surveys were "graded" out of total available points per survey to determine "humane" scores. Two surveys were made: one for younger children, one for older children. I would like to determine whether there is a significant difference between the before and after scores for the young children, before and after scores for the older children, and before and after scores for all children.

The main issue I'm having is due to the fact that my sample sizes are not equal in the before and after categories, for younger, older, AND total. Some parents rushed their kids right out after the program and I wasn't able to get their scores. Does this have any bearing?

I really don't have an issue with performing or interpreting the tests, but choosing one is my downfall. I just don't know where to start!

Thanks for any help or suggestions!
 

trinker

ggplot2orBust
#2
This sounds pretty interesting. The rushed out kids are missing data and that's part of the game. Yes this could have a bearing on your report. You could choose any number of ways to deal with this problem. The easiest is to delete them from the study altogether, provided they're missing at random and there's not too many of them. You can also choose other means of imputation (SEE THIS LINK).

Now as far as which test (first I assume you have matched data meaning you can match the pre score for one kid to his post score):

You're interested in a change after treatment (seeing the zoo). You are after some repeated measures approach. On method is to use difference scores (difference between pre and post scores). Another is a repeated measures anova. Still another an ancova with the pre score as a covariate. I'd personally approach this with an Ancova. Put the post score as outcome (DV) and then enter the pre in first as a covariate and then age as a predictor.
 
#3
Unfortunately, the data is not matched. I allowed the children to mark their answers with different colored crayons (their choosing) so I MAY be able to remove the appropriate pre-scores. The issue with this is the fact that my sample sizes are very small (9 young before, 8 young after, 5 old before, 3 old after). I definitely realize that this will have major effects on my outcomes, but it's what the zoo gave me to work with.

If I remove the unpaired data, I can just do a repeated measures (paired) t-test, correct? Again, I'm comparing young before/after, old before/after, and total before/after. I'd like to make this as simple as possible for all our sakes. I'm not exactly math-minded (I can perform the calculations and vaguely explain the results, but that's as far as I go) and the folks at the zoo are even farther removed from the math realm than I am. Eep.
 
#4
I guess I wasn't very clear when I was asking before: does the difference in sample size dictate which test to use? Do you suggest a certain test for small AND unequal sample sizes? Is there a test I can use that takes this into account?
 

trinker

ggplot2orBust
#5
Differences in sample size isn't really a concern with most test statistics as modern programs automatically do the weighting if required for each group.