Non Inferiority with a Historical Control

Is it possible or valid to do a Non-Inferioirty Test againts a historical control. I should point out for the historical control all I have is a published point estimate and a 95% CI, and a sample size. Raw data is not available.

Let me try to explain the context a little. Treatment A has been modified to Treatment B. Treatment A is no longer available. We have some data that shows B is non-inferior to A in terms of clinical effect. But B may produce more adverse events(AEs). AEs were not captured adequately in the non-inferiority study, and now Treatment A is no longer available. But we do have a fairly good study that says 98.9% of patients (95% CI, 99.7 - 97.2) didn't experience the specific adverse event we are interested in. If we were able to do a RCT we'd set the non-inferiority margin at 0.85.

But we can't run an RCT as Treatment A simply can't be recreated any more.

I guess this type of question must come up for epidemiological studies a lot? But perhaps they don't call it non-inferiority and thats why I can't find anything about non-inferiority published that refers to using historical control data? Or maybe people have better access to original raw data?

At the moment, all I'm trying to do is a sample size calaculation to understand, if we were to Audit Treatment B AEs - how big a sample would we need to demonstrate that Treatment B isn't any worse than treatment A.


Omega Contributor
Yes, you can do this. The key limitation is that treatments were not randomized, so you can't completely validate samples are comparable and there aren't any unknown or omitted variables that may confound results. If you had baseline covariates for both samples you could have controlled for them or used propensity scores, but you seem to have your hands tied.

Also, you don't know if the parameters in the historic data were completely appropriate. Meaning the original authors may have reported means, standard deviation and say t-tests, but these data could have been skewed, had an outlier or influencers, and you would not completely be privy to the true nature of those data. You can probably tell given the measures of central tendency and dispersion if there are any blatant signs of data distribution issues.

P.S., You would also want to keep in mind any historic bias, say changes over time in say available drugs, products, techniques, insurance coverage, pretty much any change that occurred due to the passing of time that would make the two periods not completely comparable.