Overlapping sample: t test with partial pairing??

[JAaron]

Hi -

I've a work project that requires a stat test between Product A and Produce B performance ratings (On a scale of 1 to 5 how well did Product A / B perform?)

We are treating the data as interval scaled.

The samples partially overlap. e.g. Product A has n=500; Product B has n=500; there are 250 common respondents between the two.

What test is appropriate here?

I dug around and the only detailed reference I could find that seemed to fit the bill is here on page 20. Dependent t-test with partial pairing. Is that what I need?
http://www.analyticalgroup.com/statistical_reference14/Statistical Reference.pdf

Thank you!

 

[Jake]

A mixed model / multilevel model / hierarchical linear model / random effects model (this same model goes by many different names, as you can see) can easily handle this.

 

[CB]

Jake, just out of interest, can I check how you would specify the mixed model? I'm thinking something just like:

lme(Performance ~ Product, random = ~1 | Respondent)

 

[Dason]

Alternatively if you didn't want to impose a normal distribution on the respondent effects you could use a linear model just using the response as is for respondents that only did either A or B and using the difference of the scores (A - B) as the response when you have both for a respondent. Then if you code the X matrix right you'll be able to get estimates for the means for each product and also the estimate of the difference in the means.

There are issues with both approaches. The mixed model will allow you answer more questions (some of which you might not care about) but also adds some assumptions to your model. You already mentioned you're dealing with data on a scale of 1-5 and this isn't ideal for either approach but with the sample size you have should probably be fine.

 

[Jake]

@CB, Yeah pretty much.

@Dason, Could you maybe illustrate how you would set up such a model, i.e., the X matrix?