Summarizing Treatment Effects from a Number of Studies?

#1
Hi all,

Thanks for taking the time to read my question. Apologies in advance for the likely basic nature of my question (its been a few years since Ive done any proper statistics).

In short, Im in the process of writing a book about the various treatments for Crohn's Disease, and Im looking for a way to summarize the treatment effects of various studies. As an example, a bunch of studies compare Remicade to placebo and each give summary statistics like: "60% of patients receiving Remicade went into remission vs 20% in controls at week 4". Whats the best way to combine the treatment effects of a number of studies like this to give the reader a rough idea of the "true" treatment effect?

Thanks for the help!
 

hlsmith

Omega Contributor
#2
I didn't think books introduced new pooling of data. What you are thinking of is a meta-analysis. What does the Cochrane review say on the topic? That would be your gold standard for evidence. If for some reason their isn't a good Cochrane review, which I would find hard to believe, let me know and we can publish a meta-analysis and you would cite that!
 
#3
I didn't think books introduced new pooling of data. What you are thinking of is a meta-analysis. What does the Cochrane review say on the topic? That would be your gold standard for evidence. If for some reason their isn't a good Cochrane review, which I would find hard to believe, let me know and we can publish a meta-analysis and you would cite that!
Thanks for the reply! In general books dont tend to introduce new pooling data so youre right about that, but I didnt do a proper systematic review and rather hand picked (i.e. large potential for bias) about 50 of the most significant studies over the last 15 years, so it wouldnt be a result one could publish in the medical literature anyway. My intended audience is patients and medical students/residents, rather than practicing gastroenterologists, and the goal for this portion is really just to say, "Youve got a ~58% chance of going into remission at week 8 with the drug", for example.

Ive looked into summary statistics for meta-analyses and from what I understand there are two broad options: dichotomous outcomes where Id need to use OR and/or RR and continuous data where Id use mean differences and/or standardized mean differences (SMDs). Ive got the # of events and population size for all the RCTs, but Id rather not present data as ORs and RRs for the layperson if possible. On the other hand, SMDs seem far worse (thankfully all the studies use the same "scale"), and Id rather talk just about the effect in the treatment group rather than the difference between the treatment and control group (which I believe would represent the "mean difference").

Tl;dr - Im trying to see if there is a way to combine the following sort of data:
  • Study 1: n = 108, remission in treatment = 60%
  • Study 2: n = 53, remission in treatment = 54%
  • Study 3: n = 334, remission in treatment = 81%

Tangential question: if my outcome is a binary one (remission + or remission -), but its presented as a mean (i.e. 60% of patients in this group achieved remission), would I be dealing with dichotomous outcome data or continuous data?
 

hlsmith

Omega Contributor
#4
Pooling studies is a tedious and methodical practice, not crazy hard but very protocolized. You have to look to see if they have comparable study designs (e.g., RCT, observational controlling for covariates)? Then you have to control for heterogeneity, sample size, publication bias, etc. You are probably just best off just presenting a big table where the reader can digest the varied results.

ORs usually presented in retrospective studies or those that don't establish temporality. One would have to see how that particular study analyzed/presented data.
 
#5
Pooling studies is a tedious and methodical practice, not crazy hard but very protocolized. You have to look to see if they have comparable study designs (e.g., RCT, observational controlling for covariates)? Then you have to control for heterogeneity, sample size, publication bias, etc. You are probably just best off just presenting a big table where the reader can digest the varied results.

ORs usually presented in retrospective studies or those that don't establish temporality. One would have to see how that particular study analyzed/presented data.
Would a weighted average not be sufficient for the "unscientific" nature of the book? I originally was just going to eyeball it (all the studies are within the 50%-65% success range with near identical protocols) but Id rather have at least SOME kind of a logical reason to it. It seems that they like to use inverse variance weighting in most meta-analyses, but for the % data that Im collecting, they obviously dont provide Std, CIs etc, so all I can really do is weight by sample size. Would that be an close enough approximation for my purposes?

Also: if my outcome is a binary one (remission + or remission -), but its presented as a continuous one (i.e. 60% of patients in this group achieved remission), would I be dealing with dichotomous outcome data or continuous data for the purposes of analysis?
 
#6
I think Ive figured out the reason for my confusion, but please correct me if Im wrong. For a meta analysis, we can only use "effect size" data, which I believe are only the "mean difference", "standardized mean difference", ORs, and/or RRs. Essentially quantifying the magnitude of the difference between treatment and control groups. Eg. 70% remission in treatment group vs 30% remission in control group would give a "mean difference" of 40%, which could then be used in a MA.

However, for MY purposes, Im only interested in finding a "mean" of percentages in the treatment group (i.e. disregarding the control group entirely). Is there any way of statistically combining those percentages?
 

hlsmith

Omega Contributor
#7
Can you just post a link to the referenced paper along with directions to the page, column, line you are referring to.

Yes, most MAs are based on effect size measures. You can convert between them, but in my opinion precision gets lost in regards to the estimate itself.
 
#8
Can you just post a link to the referenced paper along with directions to the page, column, line you are referring to.

Yes, most MAs are based on effect size measures. You can convert between them, but in my opinion precision gets lost in regards to the estimate itself.
Well I would need to post a number of different studies since Im looking to summarize/pool the rates in the treatment groups from multiple studies. Like in the example I gave above (shown here):
  • Study 1: n = 108, remission in treatment = 60%
  • Study 2: n = 53, remission in treatment = 54%
  • Study 3: n = 334, remission in treatment = 81%

The alternative would be to do a random-effects MA for continuous data using the "mean" differences (treatment vs control groups), but since %s dont come with SE or Std (and %s obviously arent means), I dont think that that is even possible.

Is there no way, then, to say that someone has a (pooled/summarized) XX% of going into remission on drug Y? Because thats all Im really looking to be able to say.
 

hlsmith

Omega Contributor
#9
Are studies randomized, control for different covariates, or have any systematic differences?

Why can't you just say results varied from 54-80% for remission? Also I was curious to peruse one of the papers since remission is a time bounded outcome. Was it remission within 1-year with right censored data - did all studies have equivalent follow-up? See works by Miguel Hernan for follow-up length concerns.
 
Last edited: