+ Reply to Thread
Results 1 to 8 of 8

Thread: How to test baseline differences?

  1. #1
    Points: 7,116, Level: 55
    Level completed: 83%, Points required for next Level: 34

    Posts
    35
    Thanks
    0
    Thanked 0 Times in 0 Posts

    How to test baseline differences?




    I have 5 different groups, each with their own range of data elements like demographics and other metrics. I was wondering if it makes sense to test for baseline differences (i.e., just differences between these 5 groups, like differences in demographic variables or other metrics) by doing a t-test or one way ANOVA.

    I am not doing an RCT or comparing intervention group versus control group. I just have 5 different groups and want to see if there are any differences between them

    New to stats and not sure if I can actually do this. Is there anything wrong with my approach?

  2. #2
    Points: 7,116, Level: 55
    Level completed: 83%, Points required for next Level: 34

    Posts
    35
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: How to test baseline differences?

    No one can help with this??

  3. #3
    TS Contributor
    Points: 17,749, Level: 84
    Level completed: 80%, Points required for next Level: 101
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,540
    Thanks
    56
    Thanked 640 Times in 602 Posts

    Re: How to test baseline differences?

    What groups are these, how are they defined? How were subjects allocated to groups? How large are the groups? What is your study about, what is the research question? Which role do the groups play/does the grouping of subjects Play in your study? Why are you interested in baseline differences?

    With kind regards

    Karabiner

  4. #4
    Points: 7,116, Level: 55
    Level completed: 83%, Points required for next Level: 34

    Posts
    35
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: How to test baseline differences?

    The groups are actually data collected from different sites. The subjects weren't allocated to the sites, they are just clients who have come in and completed a survey. the groups or sites range from 20-100 ppl in each site. There is no research question at this point. We just want to see what differences (if any) there are at baseline (like demographic data) to see if this can give us any clues as to what might be causing differences in outcomes we are seeing. But, to be clear we are not going to say that any differences at baseline CAUSE the differences in the outcomes we are seeing.

  5. #5
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: How to test baseline differences?

    "But, to be clear we are not going to say that any differences at baseline CAUSE the differences in the outcomes we are seeing." So this variable will not be tested at all, correct? Or by "CAUSE" you are saying that you wont many causal conclusions? Step one if you are going to look at that variable, just create a boxplot for the five groups and scores or a scatterplot with groups results clustered.


    So you will just compare say percent female between 5 groups: 1v2, 1v3, 1v4, 1v5, 2v3, 2v4, 2v5, 3v4, 3v5, and 4v5. Do you see any reason why this may be a less than ideal idea (hint the number of comparisons and sample sizes)?
    Stop cowardice, ban guns!

  6. #6
    Points: 1,741, Level: 24
    Level completed: 41%, Points required for next Level: 59

    Posts
    230
    Thanks
    37
    Thanked 68 Times in 59 Posts

    Re: How to test baseline differences?

    It kind of sounds like the OP is asking how to test for covariate balance in non-randomized groups to evaluate potential sources of confounding when analyzing the dependent variable (or ID variables that might need to be included in a modeling procedure to mitigate confounding). Is this correct, OP?

  7. #7
    TS Contributor
    Points: 17,749, Level: 84
    Level completed: 80%, Points required for next Level: 101
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,540
    Thanks
    56
    Thanked 640 Times in 602 Posts

    Re: How to test baseline differences?

    Does one need to formally test that? If sample size is large, irrelevant confounders may become
    statistically significant. If sample size is low, then power is low, and even important imbalances between
    groups might not turn out "significant".

    With kind regards

    K.

  8. #8
    Points: 1,741, Level: 24
    Level completed: 41%, Points required for next Level: 59

    Posts
    230
    Thanks
    37
    Thanked 68 Times in 59 Posts

    Re: How to test baseline differences?


    Quote Originally Posted by Karabiner View Post
    Does one need to formally test that? If sample size is large, irrelevant confounders may become
    statistically significant. If sample size is low, then power is low, and even important imbalances between
    groups might not turn out "significant".

    With kind regards

    K.
    It's a common practice in health related research to do this. Ideally, people will use their judgement and theory to decide how to interpret a significant "imbalance." Some people will include these potential confounders in the modeling process to judge the effect size to see if it's really a concern for confounding, or, more objectively, the model is fit with and without a confounder and %change in the beta estimates are checked to determine potential confounding issues (suggestions from Hosmer & Lemeshow texts).

    I've actually seen research in top journals use this practice even in randomized controlled trials. Theoretically, I would think you wouldn't check for covariate balance in cases of randomization, so I spoke to a biostatistician (PhD in Statistics). His advice was that you shouldn't have to worry (theoretically), but it's not a bad idea to check for balance because even a random allocation can turn out unbalanced, and can still cause confounding in some cases (which made sense to me after his explanation, but some people do disagree with the approach, although I don't know their background in statistical theory and practice). He also suggested running models with and without the potential confounders to determine the robustness of the conclusions and estimates (as mentioned above).

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats