+ Reply to Thread
Results 1 to 8 of 8

Thread: Unpaired repeated measures test?

  1. #1
    Points: 486, Level: 9
    Level completed: 72%, Points required for next Level: 14

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Unpaired repeated measures test?




    My dataset is pre- and post-test Likert Scale around an intervention, but it is not matched: it is collected anonymously from the same cohort of students (therefore I assume some small change in who's in class on each testing day).

    I am struggling with the analysis of this data. I can group responses from the scales and compare proportions pre- and post-, but haven't figured out the best way to compare full responses. Chi-square, Wolcoxon, t-tests, etc. all require, as I understand it, independent groups, which repeated measures are not.

    I am also not clear on how to approach the idea of summing several responses and running ANOVAs, or if that's even possible.

    It's amazing how much information is online about how to do all these tests, but there appears to be a dearth of information about the details of selecting appropriate tests!

    Thank you all for thoughts and feedback,

    Peter

  2. #2
    Points: 1,362, Level: 20
    Level completed: 62%, Points required for next Level: 38

    Posts
    49
    Thanks
    3
    Thanked 1 Time in 1 Post

    Re: Unpaired repeated measures test?

    You could just perform a paired t-test (or equivalently one sample t-test on the change scores (post-pre)) to see if the mean change is significantly different from 0. If a parametric test isn't appropriate (if you have a small sample size and data are far from normal) you could use the Wilcoxon test instead.

  3. #3
    Points: 486, Level: 9
    Level completed: 72%, Points required for next Level: 14

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Unpaired repeated measures test?

    Quote Originally Posted by kabooze View Post
    You could just perform a paired t-test (or equivalently one sample t-test on the change scores (post-pre)) to see if the mean change is significantly different from 0. If a parametric test isn't appropriate (if you have a small sample size and data are far from normal) you could use the Wilcoxon test instead.
    But I have been told that a paired t-test is only valid if the two groups are independent. Both my groups are the same people, simply tested at different times, so they're not independent. Am I misunderstanding?

    Thanks for your thoughts!

  4. #4
    Devorador de queso
    Points: 95,540, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,930
    Thanks
    307
    Thanked 2,629 Times in 2,245 Posts

    Re: Unpaired repeated measures test?

    Well you probably misunderstood whoever told you that because the only time a paired t-test is appropriate is if there is some sort of correlation between the observations (as in the case where you have two observations on the same person).

  5. #5
    Points: 338, Level: 6
    Level completed: 76%, Points required for next Level: 12

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Unpaired repeated measures test?

    I have been struggling with the same question. I have "repeated" data, in that I have repeated measures on the same group. However, I cannot "pair" the data due to the nature of the test (like the anonymous scores here. I'm not sure which score goes with which individual). Would an ANOVA (I have more than two time points) with a repeated measure correction be appropriate?

  6. #6
    Cookie Scientist
    Points: 13,431, Level: 75
    Level completed: 46%, Points required for next Level: 219
    Jake's Avatar
    Location
    Austin, TX
    Posts
    1,293
    Thanks
    66
    Thanked 584 Times in 438 Posts

    Re: Unpaired repeated measures test?

    No. You cannot do a repeated measures analysis if you don't know which measurements are the repeated ones for each unit. I'm afraid the best you can do is to pretend that your pre- and post-scores are independent and do a regular t-test. Unfortunately this test will be biased in the direction of being overly conservative.
    “In God we trust. All others must bring data.”
    ~W. Edwards Deming

  7. #7
    Points: 2, Level: 1
    Level completed: 3%, Points required for next Level: 48

    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Unpaired repeated measures test?

    Peter: I am in the same quandary. Did you figure out the way forward with this? I would be really interested!

    Shankar

  8. #8
    Points: 3,631, Level: 37
    Level completed: 88%, Points required for next Level: 19
    staassis's Avatar
    Location
    New York
    Posts
    226
    Thanks
    2
    Thanked 41 Times in 39 Posts

    Re: Unpaired repeated measures test?


    If there is some, unknown overlap between the pre- and post- groups but the subjects of the groups are not matched, then BOTH independent samples t-test and paired t-test are not valid... However, there is a way out. Use PROPENSITY SCORE MATCHING based on many other variables in the data set to match a subset of subjects in the pre- group to a subset of the subjects in the post- group. Then run paired t-test on the two matched subsets.

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats