Would this have an effect on study results regarding statistics?

Dear all,
I am in need of urgent brainstorming for my study design.
To cut it short here is the scene:

There is this study where I need to do some cognitive tests on patients (before /after treatment) and their native language might affect the results of these tests. If I give this tests in English they would probably score less than what they would normally score. My colleague argues that this would not have any effect on the outcomes in general since we are comparing the before and after states anyway.
But these patients would enter the study with lower scores. So I am in favor of including only native speakers.
Any ideas?


Less is more. Stay pure. Stay poor.
Well, results can only be generalized to comparable samples of patients. Are scores bounded, meaning they are contain within say 0-100%? Another threat is effect modification, that would be that the intervention or change in scores would not be comparable when stratifying by groups. For example, perhaps the non-native speakers perform poorly on pre-test and better on the post-test, partially based on introduction and now familiarity with testing language, not solely because of the intervention. In addition, if scores are bounded, native speakers may not have as much room for improvement if they performed well on the pre-test compared to non-native speakers.
Last edited:
Use a multilevel model (or categorical) covariates where you can estimate the the effect size compared to a reference. This way, we can compare both in the same model. If we use partial pooling, we can gain information from both groups.