Control group


No cake for spunky
In theory we would like to know those who got our service results compared to those who did not. And we would like to randomly assign people to services. The latter is not legal and we do not have information on those who are similar to our customers, but got no services from us. So while that limits our analysis we are stuck.

So given that, how invalid is it to analyze the impact of getting a service by looking at our customers who get the service and those that did not controlling for factors like gender that impact result. All we want to know is, did a specific service or behavior or ours have an impact. Of course analyzing the relative impact of getting service 1 compared to service 2 would be awesome, but after ten years of looking for ways to do that (different issue)I don't think there is an agreed way to do that :)


Active Member
Sounds fine to me. In my work, we roll out services to a small segment of the customer base before expanding access to everyone. And we analyze customers with the service against those without the service all the time. We try to define the "control" group as close to the test group minus the service.


TS Contributor
I do not fully understand the situiation. You say "looking at our customers who get the service and those that did not".
So you have follow-up data on those who did not receive the service(s)?

What is the analysis ment for, e.g. internal quality assessment? To justify the program towards funding agencies?
Scientific achievement.

What additional information do you have about customers? Might propensity scoring be an option?

With kind regards



Less is more. Stay pure. Stay poor.
@noetsi do you have the "got a job or not, income gains" only for those receiving services? @Buckeye - he is writing about state services, so I am guessing the state can't withhold service. Though I would imagine if they had new services, they could "roll them out" or have a before and after.

Can you compare your data to other states? Or since you live in FL, third world countries. Just kidding.