# Thread: How to test baseline differences?

1. ## How to test baseline differences?

I have 5 different groups, each with their own range of data elements like demographics and other metrics. I was wondering if it makes sense to test for baseline differences (i.e., just differences between these 5 groups, like differences in demographic variables or other metrics) by doing a t-test or one way ANOVA.

I am not doing an RCT or comparing intervention group versus control group. I just have 5 different groups and want to see if there are any differences between them

New to stats and not sure if I can actually do this. Is there anything wrong with my approach?

2. ## Re: How to test baseline differences?

No one can help with this??

3. ## Re: How to test baseline differences?

What groups are these, how are they defined? How were subjects allocated to groups? How large are the groups? What is your study about, what is the research question? Which role do the groups play/does the grouping of subjects Play in your study? Why are you interested in baseline differences?

With kind regards

Karabiner

4. ## Re: How to test baseline differences?

The groups are actually data collected from different sites. The subjects weren't allocated to the sites, they are just clients who have come in and completed a survey. the groups or sites range from 20-100 ppl in each site. There is no research question at this point. We just want to see what differences (if any) there are at baseline (like demographic data) to see if this can give us any clues as to what might be causing differences in outcomes we are seeing. But, to be clear we are not going to say that any differences at baseline CAUSE the differences in the outcomes we are seeing.

5. ## Re: How to test baseline differences?

"But, to be clear we are not going to say that any differences at baseline CAUSE the differences in the outcomes we are seeing." So this variable will not be tested at all, correct? Or by "CAUSE" you are saying that you wont many causal conclusions? Step one if you are going to look at that variable, just create a boxplot for the five groups and scores or a scatterplot with groups results clustered.

So you will just compare say percent female between 5 groups: 1v2, 1v3, 1v4, 1v5, 2v3, 2v4, 2v5, 3v4, 3v5, and 4v5. Do you see any reason why this may be a less than ideal idea (hint the number of comparisons and sample sizes)?

6. ## Re: How to test baseline differences?

It kind of sounds like the OP is asking how to test for covariate balance in non-randomized groups to evaluate potential sources of confounding when analyzing the dependent variable (or ID variables that might need to be included in a modeling procedure to mitigate confounding). Is this correct, OP?

7. ## Re: How to test baseline differences?

Does one need to formally test that? If sample size is large, irrelevant confounders may become
statistically significant. If sample size is low, then power is low, and even important imbalances between
groups might not turn out "significant".

With kind regards

K.

8. ## Re: How to test baseline differences?

Originally Posted by Karabiner
Does one need to formally test that? If sample size is large, irrelevant confounders may become
statistically significant. If sample size is low, then power is low, and even important imbalances between
groups might not turn out "significant".

With kind regards

K.
It's a common practice in health related research to do this. Ideally, people will use their judgement and theory to decide how to interpret a significant "imbalance." Some people will include these potential confounders in the modeling process to judge the effect size to see if it's really a concern for confounding, or, more objectively, the model is fit with and without a confounder and %change in the beta estimates are checked to determine potential confounding issues (suggestions from Hosmer & Lemeshow texts).

I've actually seen research in top journals use this practice even in randomized controlled trials. Theoretically, I would think you wouldn't check for covariate balance in cases of randomization, so I spoke to a biostatistician (PhD in Statistics). His advice was that you shouldn't have to worry (theoretically), but it's not a bad idea to check for balance because even a random allocation can turn out unbalanced, and can still cause confounding in some cases (which made sense to me after his explanation, but some people do disagree with the approach, although I don't know their background in statistical theory and practice). He also suggested running models with and without the potential confounders to determine the robustness of the conclusions and estimates (as mentioned above).

 Tweet

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts