Hello all,

I've been asked to assist with the statistical analysis of some survey data, and I'm a bit of a novice with this type of analysis, so I'm hoping that some with more experience can give me a bit of guidance.

The investigators are interested in assessing the effectiveness of 6 different videos. They have shown all of the videos to each of 60 volunteers. The volunteers were then given a questionaire and asked to answer the same 6 questions in relation to each video; the questions were about various aspects of the video, with the last question being essentially "Give an overall ranking". The possible answers to each question were on a scale of 1 to 5, with 5 being "extremely effective" and 1 being "extremely ineffective" (except for one question, which for some reason was simply from 1 to 3). Finally, the participants were asked to rank the videos from 1 to 6.

My thinking is as follows, and please correct me if I'm wrong. There are only 6 questions, so I might as well analyze each question separately, and then analyze the final "Rank the videos" question on its own. The data is ordinal, so I should use nonparametric methods. Beyond that, I'm a little unsure of myself, and here are my questions.

1. The chi-square test comes to mind, but I'm not quite sure how to set it up. One thought is to do a 2 variable chi-square, with the two variables being "answer on the question", the other being "video number". This would then have (6-1)(5-1) = 20 degrees of freedom, but I am a little worried as to the high number of the degrees of freedom vs. the relative smallness of the sample. Do I need to make sure each bin has at least 5 entries in it, or something?

2. If the chi-square test above reports significance, how can I tell which video performed better than the others? One idea was to add up the answers to all the videos, so this would give me 6*50 = 300 answers allocated to the bins 1 - 5, and this would give me the expected distribution of the answers. Then I can compare this to each video individually, dividing the numbers above by 6 to get the expected value in each bin, and then do a chi-square with the observed vs. the expected (so now 4 degrees of freedom). I think this would allow me to compare each video against the averages. Does this seem like a reasonable way to do it?

3. I need a different way to analyze the "Rank the videos" question, maybe. I guess I could again do a 2 variable chi-square with the ranking that each video gets as the answer to that question, but I feel like there must be a different, better way to do this one.

4. Finally, although it's not relevant to this study since I already have the data, I can't help but be curious over the following question related to survey design. It seems to me that the order in which the videos are watched will have an effect on the answers, so it would be good to arrange those in such a way that the effect of this is minimized. I was thinking that, suppose we had 60 participants, let's give them each a different order, but with the property that each video gets viewed first 10 times, then second 10 times, etc. Also maybe each video should follow each of the others 10 times, or something. Is there a standard way of mixing up the order to try to make it as fair as possible?

Many thanks for any advice given. I do have access to SPSS, so I suspect it'll be pretty easy to get the computer to do the work, I just want to understand what I'm doing.

Greg