Correcting for a confounding factor in a retrospective laser eye surgery study

[eyedocdan]

Hello,

I would really appreciate some help on study design. I will present my ideas and if the forum could critique/advise I would be very grateful.

The study compares a new refractive laser eye surgery technique (lets call this Rx2) with the standard treatment (Rx1). It is already known that they perform as well as each other at giving a good visual outcome. However, it is anecdotally recognised that the new treatment (Rx2) initially leaves the patient with poorer vision, at day 1 and likely still at day14 after treatment. Finally, it is also known that pre-existing refraction (eg long or short sightedness) can effect laser refractive surgery outcomes. This is a potentially confounding factor that I would like to attempt to correct for in the study.

The ideal would be a prospective double blind randomised control trial, but this is not possible for me to perform. I am limited in the data available to me. It is anonymised retrospective data from a single laser refractive surgery clinic. Because most people have bilateral procedures I will randomly select only one eye so that one patient = one independent unit of data. Finally there is a manual component to accumulating all the data, it is not all in a spreadsheet so I am also limited in time as to the number of units of data I can collect, realistically 100.

There are over 700 patients who received Rx1
There are 50 patients who received Rx2

Study question:
Does Rx2 have a delayed visual recovery compared with standard Rx1?

Null hypothesis
Best corrected visual acuity (BCVA) at day 14: Rx2 = Rx1

Alternative hypothesis
BCVA at day 14: Rx2 does not equal Rx1

Idea 1
Stratify the groups Rx1 & Rx2 according to the confounding factor (pre-existing refraction) and randomly select 10 from 3 strata for Rx1 & Rx2 then perform an independent t-test on BCVA at day 14.

Idea 2
Use all 50 patients from Rx2 and randomly select 50 from Rx1 and keep my fingers crossed that the refractions have similar descriptive statistics. Then perform independent t-test.

Idea 3
Use all 50 patients from Rx2 and blindly match 50 from Rx1 according to closest refraction. Then perform a paired t-test on the matched pairs.

My instinct is that idea 1 is the best. I am grateful for any advice on whether I am transgressing any obvious statistical rules and suggestions for better approaches within the stated limitations. Thank you!
(I appreciate the data necessarily suffers from convenience bias because I have no access to any other data. Also FYI, the data does follow an approximately normal distribution.)

Cheers,
Dan

 

[hlsmith]

Well, are you fairly confident that refractory is a confounder? I don't get Idea 1, but 3 sounds reasonable to me. But once you match on refractory, you can never know its relationship. Are there any other things that you would need to control for, baseline vision, age, comorbidities, surgeon, learning curve associated with procedure, not everyone cam back exactly 14 days later?




So each patient is contributing only one eye or set of eyes to a single treatment group? Also, is there any confounding by indication, worse patients directed toward better treatment approach? If you have multiple confounders, you may need to explore something like propensity scores.


Have you done a literature review to see if anyone else has already done this study in a better way than you can, with more generalizable patient sample and a more rigorous protocol?

 

[eyedocdan]

Thanks hlsmith. I am certain refraction is a confounder, and I do not need to know it's relationship to the studied data. The other confounders are valid statements, thanks. Age was also on my mind as a confounder and I think I need to correct for this too. I am less certain about baseline vision being a confounder... I am thinking about it. The other factors are all pretty homogenous because otherwise they would not be having Rx1 or Rx2 and there is only one experienced surgeon performing all treatments. Each patient will only contribute one randomly selected eye and belongs only to group Rx1 or Rx2 (ie I discard the 'other eye' data as it is not independent).

To try and better explain idea 1:
I would take all Rx2 group and separate them by refraction into strata such as: 0 to -3 dioptres; -3 to -6 dioptres; -6 to -9 dioptres. I would then randomly select 10 patients from each strata to form a sample of 30. I would then do the same for Rx1 group. I would then have 2 groups with broadly similar refractions that could be compared with and independent t test. I think...

However, on your suggestion that idea 3 is better, I have formulated more detail on this plan. My main question is: Is it reasonable to match subjects when they have already been 'assigned' to treatments? When I have read about matching, I would read that matching happens first and then assignment to a treatment.

Here is my plan to execute idea 3.

Power calculation to detect clinical difference and how many cases needed (n).

Randomly select n Rx2 patients.
Blind self to patient data except, 3 confounding factors: Preop refraction; Age; Baseline vision
Randomly select eye.
Stratify each category so that matching can occur.

Identify all Rx1 patients.
Stratify all eyes according to the same 3 confounding categories with the same strata boundaries.
Then find n Rx1 matches to the Rx2 patients. ie being in the same strata for all 3 confounders. If multiple Rx1 patients are suitable to match, then randomly select.
Ensure no patients contribute 2 eyes (ie exclude Rx1 patients once one eye is selected by matching)

Unblind and retrieve the data for the patients included.

Calculate change in BCVA between day14 and discharge BCVA. (this is actually what I am testing for, not as in my first post).

Check for assumptions (normality and outliers).

Paired t-test.

 

[hlsmith]

You can match if you know treatment groupings. What was the logic / rationale for assigning treatments?

If you match on distribution of confounder, you may be getting close to frequency matching, where data doesn't have to be directly matched between groups.

 

[eyedocdan]

Thanks again hlsmith.
Logic/rationale for assigning treatment is a seive of suitability determined by clinician and then patient choice, after discussion. All the advantages and disadvantages are discussed, many of which are not relevant to my study question. But the discussion to decide on treatment is oviously biased by the statements of the surgeon, the doctor-patient relationship and patient choice. This is a bias that cannot be avoided without having a double blinded randomised control trial.
Your advice has been much appreciated. I plan to proceed as per idea 3 above.

 

[CB]

A couple of thoughts here, generally revolving around the idea of not throwing away information:

1) Ensuring independence of the error terms is important, but you don't need to only select one eye per person to do this (thus throwing away information). You could instead use a model that accounts for the presence of the two "repeated" measurements from the same person (e.g., a repeated measures ANOVA, or a mixed effects model incorporating a random intercept across persons).

2) Manually matching patient pairs (Idea 3) is an ok idea, but that's a pretty painful control strategy, and again involves throwing away some data (in this case, 650 or so control patients). Furthermore, the use of stratification of the potential confound leaves open the possibility of confounding effects within each stratum. I'd use this strategy only if I had no idea whatsover of the functional form of the relationship between the potential confounder and the dependent variable. Is reasonable to assume that there is some specific form of relationship between pre-existing refraction and the dependent variable? E.g., could you assume this relationship is linear? If so, you can just control for the potential confounder statistically rather than by using matching as part of your design. E.g., you could just use pre-existing refraction as a control variable in your statistical model. This is a much easier approach.

I am certain refraction is a confounder

Just to double check here... the conditions necessary for pre-existing refraction to be a confound are:
1. The potential confound pre-existing refraction is correlated with the independent variable - i.e., mean pre-existing refraction levels are different across the two treatment groups
2. The potential confound is not affected by the independent variable
3. The potential confound affects the dependent variable

It's obviously reasonable to assume that condition 2 is true, given that pre-existing refraction is measured prior to treatment. However, you haven't really said much about whether you've tested condition (1), which is easy and important to look at. I'd also wonder whether condition (3) is plausible, given that you've now said that your dependent variable is change in visual acuity scores. Sure, pre-existing refraction might affect visual acuity, but would it affect change in visual acuity over treatment? If you can assume that it doesn't, then you don't need to control for it (either by design or statistically).