# Urgent help: qusi-experimental analysis

#### omarokasha

##### New Member
Urgent help: quasi-experimental analysis

Hi,

I am working on a quasi-experimental study, with an intervention to reduce HIV stigma. A score has been constructed (scale 0-10) to measure HIV stigma among healthcare workers in two hospitals: one assigned for the intervention, and the other as control. Average stigma score was calculated in the two hospitals, before and after intervention, and we would like to see if the intervention has succeeded to reduce stigma in the intervention hospital compared to control. I am confused on which statistical test should be done, let's say we have the following table with average scores:

....................................Before......................................After....................
...........................Control............Intervention.......Control............Intervention
Average Score........4.0(a).................4.0(b)............3.8(c)...............2.0(d).....

Note that individuals after intervention, in both hospitals, are not the same ones before intervention, so it's independent on individual level, but paired on hospital level

Now, we have two issues that need to be tested:
1. first is the change within each hospital (c-a) and (d-b): for instance, in the intervention hospital, the difference between before and after scores. that would translate to %reduction of score in each hospital, and eventually whether reductions are significantly different comparing the two hospitals

2. Second, is the difference between the two hospitals, before and after intervention (b-a) and (d-c).

It seems to me there are two ways to do it:
1. conducting paired ttest to test the difference within each hospital before and after intervention, and independent ttest to test the difference between the two hospital before and after intervention.
2. Difference in differences, the tests for the two issues at one step ((d-b)-(c-a))

Last edited:

#### Martin Marko

##### New Member
Re: Urgent help: quasi-experimental analysis

Hi there,

not sure if I understand the design, but..

Hospital1 = labeled as "Control" group = got an intervention.
Hospital2 = labeled as "Intervention" group = did not get an intervention (just for mapping the pretest-postest effect without "real"-intervention effect)

Before = pre-(real)intervention in Hospital2
After = post-(real)intervention in Hospital2 (in case of Hospital1, there was just an another measurement without real intervention)

1) so there should be some kind of comparison of initial (pre-intervention) state in/between "hospital-group" (hospital factor at pre-intervention stage) -> there was no difference between Hospital 1 and Hospital 2, thus arguing for eqivalence (matched group).
2) there should be some kind of comparison of pre-intervention and post-intervention effect (intervention effect) in both levels of "hospital-group" factor (both hospitals - intervention and control)
3) there shoulôd be some kind of test for interaction between (hostiptal-group x intervention). As far as the hospital where there was an intervention did get better and the hospital without intervention did not get better, you may assume that there was probably (but not sure - not under complete control) no exogenous/confounding variable (as the level in control stood the same), but the effect occured (level in scale dropped down after intervention in hospital with intervention).

Because there were different people before and after intervention, it is between subject effect and because there were two hospitals, it was another between subject effect. So I think (because the scale is the same), there is no need to do a percentage (baseline) transformation but to use 2x2 ANOVA with two between subject factors (two way = intervention and hospital) and look frt interaction between them (that seems obvious from the sketched table you provided)

As long as there would not be a significant effect between before and after in Hospital1-control (4.0 to 3.8), you may support that there was no globaly affecting external effect or no/small hospital retest effect (the effect of retesting in the same hospital) or no/small time/spontaneus-change effect. And as long as there is an additional significant effect between B and A in Hospital2-intervented, you can support your purpose. But I think it is not an ideal design for proposing that the intervention is surely effective. Take a look for Solomon's design.

I hope I understood the design,
take care,

PS1: I would recomend you to use some kind of ordinal ANOVA or robust ANOVA, because of the 1 to 10 scale score range.
PS2: All the groups (four) should be matched in other possible explanatory variables + there should be enought participants to reduce random variable effects.

M.

Last edited:

#### omarokasha

##### New Member
Re: Urgent help: quasi-experimental analysis

Hi Martin,

thank you very much for the detailed response, I have been trying to reply several times but something went wrong with the forum...here's my 4th attempt!
I will comment on each of the points you raised (in italic), one-by-one

Hospital1 = labeled as "Control" group = got an intervention.
Hospital2 = labeled as "Intervention" group = did not get an intervention (just for mapping the pretest-postest effect without "real"-intervention effect)

It's the reverse, the control (hospital1) did not get the intervention.

Before = pre-(real)intervention in Hospital2
After = post-(real)intervention in Hospital2 (in case of Hospital1, there was just an another measurement without real intervention)

Before, pre intervention was also measured in Hospital1

1) so there should be some kind of comparison of initial (pre-intervention) state in/between "hospital-group" (hospital factor at pre-intervention stage) -> there was no difference between Hospital 1 and Hospital 2, thus arguing for eqivalence (matched group).

That has already been done, as shown in table

2) there should be some kind of comparison of pre-intervention and post-intervention effect (intervention effect) in both levels of "hospital-group" factor (both hospitals - intervention and control)

that is the problem I am trying to fix, without having to draw another table comparing pre and post-intervention by hospital group.

3) there shoulôd be some kind of test for interaction between (hostiptal-group x intervention). As far as the hospital where there was an intervention did get better and the hospital without intervention did not get better, you may assume that there was probably (but not sure - not under complete control) no exogenous/confounding variable (as the level in control stood the same), but the effect occured (level in scale dropped down after intervention in hospital with intervention).

I don't get this point, why should I test for interaction?

Because there were different people before and after intervention, it is between subject effect and because there were two hospitals, it was another between subject effect. So I think (because the scale is the same), there is no need to do a percentage (baseline) transformation but to use 2x2 ANOVA with two between subject factors (two way = intervention and hospital) and look frt interaction between them (that seems obvious from the sketched table you provided)

So if I am to use 2 by 2 ANOVA, should the table look like this:

.........control........Intervention
Before...A...................B........
After.....C...................D.......

using this annotation, what does ANOVA test (I need to understand if it works differently, compared to Difference In Differences (DID))...I am wondering why you did not comment on DID?

As long as there would not be a significant effect between before and after in Hospital1-control (4.0 to 3.8), you may support that there was no globaly affecting external effect or no/small hospital retest effect (the effect of retesting in the same hospital) or no/small time/spontaneus-change effect. And as long as there is an additional significant effect between B and A in Hospital2-intervented, you can support your purpose. But I think it is not an ideal design for proposing that the intervention is surely effective. Take a look for Solomon's design.

I agree that this is not the ideal, but that's the one they used

PS1: I would recomend you to use some kind of ordinal ANOVA or robust ANOVA, because of the 1 to 10 scale score range.

Good point, will do

PS2: All the groups (four) should be matched in other possible explanatory variables + there should be enought participants to reduce random variable effects.

How should that matching be done? could you elaborate more on this point?

Many Thanks