Those analyses are each testing different hypotheses, so I'm not sure that it's meaningful to compare their power. What is your study actually aimed at finding out?
Hello,
I am performing an analysis for a study and I have a small sample size (N = 25). There are pre and post test scores for each person on multiple instruments and constructs. I need to know which test will return the most power - a series of paired samples t-test, a mixed between/within ANOVA (gender is between groups and time is within groups), or an ANCOVA with pretest as the covariate.
Any feedback (or references) would be greatly appreciated.
Those analyses are each testing different hypotheses, so I'm not sure that it's meaningful to compare their power. What is your study actually aimed at finding out?
The hypotheses are flexible to fit each one of these situations, but mainly I am trying to compare the mean scores of a group of students (on 5 different surveys) before and after taking a class. There are a total of 25 students, so I have 50 total scores for each instrument (25 pretest and 25 post test). I also want to look at different demographic variables, such as gender, grade level, and major - each of these demographics have two levels. What I have done so far is a series of paired t-tests on pretest and post test without the demographics, and then for the demographics, just independent samples t tests on the pretest scores, and another independent samples t test on the post test scores. Of course these are not together, between groups over time. I could do ANOVA for each demographic separately over the two time periods. I could do ANCOVA with each demographic independently using pretest measurements as a covariate. The big problem is that I only have 25 records and i want to preserve as much power as possible. I do not want to do t tests because they would not answer groups over time. I could use GPower to try and figure it out. However, i thought someone here might have a ready answer - and perhaps even a reference
I appreciate any help and thanks again for your time.
As I mentioned above, the tests you are talking about are testing different hypotheses, so it's not really meaningful to compare their power.
I'd suggest making a list of specific research questions you're trying to answer, that details exactly which variables and relationships you're interested in. There seems to be a risk of getting lost in the garden of forking paths here, given the number of variables and comparisons that you could look at.
Once you have the research questions specified in more detail, the best analyses will hopefully become a lot clearer. From there you can use something like G*Power to calculate power for the analyses you're considering.
Tweet |