fixed vs mixed random effects model

#1
greetings,
I am reviewing some meta-analysis and have a question regarding fixed vs random effects models.
2 reviews have used basically the same RCT's for their meta analysis.
One uses a mixed random effects model (Dersimonian and Laird which I haven't head of) the other fixed. The mixed effect paper claims they chose this as they believed that heterogeneity of an I squared (sorry no idea how to insert superscript here) of 25% was too high to consider being chance. The other paper considered the I sq statistic significant for heterogeneity at 50% which seems more in line with other papers I have read and so used a fixed effects model where it was less than 50%. Is one more right than the other? Or is it fine to set whatever level of significance for heterogeneity you feel appropriate?
 

Jake

Cookie Scientist
#2
You should probably just use the random effects model any time the heterogeneity estimate is greater than 0. And honestly, even if the estimate is 0 it's probably still fine to use the random effects model -- it's not technically necessary there, but there isn't really any notable loss in efficiency. The point being that random effects should probably just be the default choice, and the decision to use fixed effects instead is what begs for special justification.
 

hlsmith

Omega Contributor
#3
I agree with Jake's last line, "the decision to use fixed effects instead is what begs for special justification".


Typically in meta-analysis an I^2 of 50% or greater is typically used, but you can't fault someone for saying that believed there could be heterogeneity and they are going to use random effects. Controlling for random effects serves to account for the differences between the study samples, meaning they add another piece to the SE. This translates to a lower risk of a type I error. The Sersimonian Laird approach is very common, I would say it is what is usually used, even if authors don't note its use.
 
#4
Thank you!! that certainly makes sense to me now. Regardless of the similarity of the RCT's studied it is difficult for me to really believe that any two studies are anything but heterogeneous, unless its a replicated study and even then time and center etc cant possibly be the same to any real degree.
Much appreciated
 

hlsmith

Omega Contributor
#5
Correction on my last point, random effects works to address differences in study sample, but I neglected to also mention it also needs to be implemented if study designs or procedures were different. So not just sample issues but also, just how they conducted the study.