test of significance for comparing change between two groups

[a_zensun]

What test for significance is best to use to test the difference between the change of a categorical variable in a control group and the change in a pilot group?

I do not think I can simply compare the 'after' for each because the 'before' for the two groups was statistically different - they did not start out the same so I believe I can only analyze the change within each group.

I used the two-tailed, two-proportion z-test for within group differences.

 

[Karabiner]

What test for significance is best to use to test the difference between the change of a categorical variable in a control group and the change in a pilot group?

What kind of variable, what kind of change(s)?
A binary variable? Or one with > 2 categories?

One general approach could be to classify changes
(e.g. for a binary viariable: 0=>1 / 1=>0 / no change)
and compare the classified changes (leaving out the
"unchanged" class) using two samples Chi² test.

With kind regards

K.

 

[a_zensun]

The variable had three categories (e.g. on-street parking, off-street parking, and garage parking). We want to see if the change of the percent of people who utilized each of these different types of parking in the our control area was statistically different than the change of the percent of people who utilized each in out pilot area.

As an example:

Parking on-street:

Pilot before = 209/737 = 28%
Pilot after = 145/754 = 19%
Net percentage point change = -9%

Control before = 297/713 = 42%
Control after = 242/716 = 34%
Net percentage point change = -8%

Was the change in the pilot area statistically different from the change in the control area?

Thanks!!!!

-S

 

[Karabiner]

So, is the each sample the same before and after
(differing numbers only due to missing observations)?
If yes, were data gathered on an individual level so
you can identify an individual's behaviour before and
after? Or do you only have aggregate data?

With kind regards

K.

 

[a_zensun]

Please forgive me if I do not use the correct terminology...

There were 4 different surveys taken (before pilot, before control, after pilot, and after control) - hence, the 4 different sample sizes. The same individuals were not surveyed in the before and after groups.

 

[Karabiner]

So you have some participants twice in the data set, some only
once. I am not sure how to deal with such situations properly.

If each of the observations was from a different subject AND you
could break down your research question(s) to binary ones
(e.g. effect of the intervention on whether someone did street
parking yes/no), then a binary logistic regression could be
used, with 3 predictors: when was the observation (before/after),
where was the observation (experimental area/control area),
and the interaction of these two. The interaction would tell you
whether the before-after difference was different between areas.
But I don't know how the test results are affected by the fact
that not all observations are from different subjects. Maybe someone
else here has an idea.


With kind regards

K.

 

[Mean Joe]

I do not think I can simply compare the 'after' for each because the 'before' for the two groups was statistically different - they did not start out the same so I believe I can only analyze the change within each group.

I'm wondering if you should subset the groups to make them similar? Or what statisticians call(?) matching.

What is the size of your samples? How different are the samples?