+ Reply to Thread
Results 1 to 4 of 4

Thread: Which statistical test will return more power??

  1. #1
    Points: 10, Level: 1
    Level completed: 19%, Points required for next Level: 40

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Which statistical test will return more power??




    Hello,

    I am performing an analysis for a study and I have a small sample size (N = 25). There are pre and post test scores for each person on multiple instruments and constructs. I need to know which test will return the most power - a series of paired samples t-test, a mixed between/within ANOVA (gender is between groups and time is within groups), or an ANCOVA with pretest as the covariate.

    Any feedback (or references) would be greatly appreciated.

  2. #2
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: Which statistical test will return more power??

    Those analyses are each testing different hypotheses, so I'm not sure that it's meaningful to compare their power. What is your study actually aimed at finding out?

  3. #3
    Points: 10, Level: 1
    Level completed: 19%, Points required for next Level: 40

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Which statistical test will return more power??

    The hypotheses are flexible to fit each one of these situations, but mainly I am trying to compare the mean scores of a group of students (on 5 different surveys) before and after taking a class. There are a total of 25 students, so I have 50 total scores for each instrument (25 pretest and 25 post test). I also want to look at different demographic variables, such as gender, grade level, and major - each of these demographics have two levels. What I have done so far is a series of paired t-tests on pretest and post test without the demographics, and then for the demographics, just independent samples t tests on the pretest scores, and another independent samples t test on the post test scores. Of course these are not together, between groups over time. I could do ANOVA for each demographic separately over the two time periods. I could do ANCOVA with each demographic independently using pretest measurements as a covariate. The big problem is that I only have 25 records and i want to preserve as much power as possible. I do not want to do t tests because they would not answer groups over time. I could use GPower to try and figure it out. However, i thought someone here might have a ready answer - and perhaps even a reference

    I appreciate any help and thanks again for your time.

  4. #4
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: Which statistical test will return more power??


    Quote Originally Posted by Brianna View Post
    i want to preserve as much power as possible.
    As I mentioned above, the tests you are talking about are testing different hypotheses, so it's not really meaningful to compare their power.

    I'd suggest making a list of specific research questions you're trying to answer, that details exactly which variables and relationships you're interested in. There seems to be a risk of getting lost in the garden of forking paths here, given the number of variables and comparisons that you could look at.

    Once you have the research questions specified in more detail, the best analyses will hopefully become a lot clearer. From there you can use something like G*Power to calculate power for the analyses you're considering.

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats