Comparing data between two surveys.

#1
Hi,

In my semester project I have made 2 different surveys. The first was send out in the beginning of April, and the other yesterday.

I would like to compare my results both internally in the surveys, which does not seem to be that big of a problem, but also between them, which gives me quite the headache.

The problem is that some of the respondents in survey 2 have also answered the first survey. I know this because there is a question in survey 2 asking whether or not the respondent have answered the first one as well, which some have, and some have not.
In the second survey I know which of the respondents have answered the first one as well, but I have no way of determining which of the respondents in the first survey also answered the second survey.

This means that I have no idea of how to compare the surveys statistically. I have looked at comparing the surveys with a mixed design, but I am not sure if this will work, due to the fact that between the surveys some respondents are within and some are between.

All the data I have to analyze statistically is interval data, and if it meets the assumptions I'm going to use paired t-test for some of the questions, and ANOVA for the rest, for the analysis within each of the surveys.

I haven't gathered all my data from the second survey, yet.

I really hope someone can help me.

Regards
Cecilie
 

hlsmith

Less is more. Stay pure. Stay poor.
#2
If you cannot match the people and you have a mix of those that did and did not complete both, you likely cannot make comparisons, since the samples are not independent and the dependency cannot be discerned.
 

noetsi

No cake for spunky
#3
A qualitative analysis would not make any statistical assumptions as far as I know. Including independence. So you can do text analysis if there is any.
 

hlsmith

Less is more. Stay pure. Stay poor.
#4
@Cecilie - your scenario is not a novel one. I feel as though I see it most often in pre-post intervention scenarios where the two sample have partial overlap. The major concern is usually that the most extreme people are not in both samples. Thus there is a systematic loss in people that can mislead results.

I would imagine that this would also be a design flaw for qualitative analyses as well.
 

noetsi

No cake for spunky
#5
Qualitative text analysis really make no statistical assumptions. Some of the more extreme in that field in fact attack quantitative assumptions although I think that is not the norm for the field.

For instance there is no minimum sample size, sort of, no assumption of independence that I am aware of. It is about making judgements of whether you have learned something useful or not.