Differences in responses on a 5 point scale between three groups? Kruskal-Wallis?

#1
Dear madam/sir,
Recently I have designed a course that was run at three different locations. To know what the students thought of the course they were given the same questionnaire on all three locations. One of the questions as an example was to indicate the extend to which they agreed with the statement: The teacher motivated me. Fully disagree O O O O O Fully agree.
The individual options 'Fully disagree', 'Disagree', 'Neither agree nor disagree', 'Agree' and 'Fully agree' were also shown at the start of the questionnaire.
On location A 17 students completed the survey, on location B 22 students and on location C 16, creating a total of 55.

Now I'm interested to know if there is a significant difference among any of the locations how the students thought about the teacher motivating them. Since the Location is nominal and the Teacher is ordinal (or nominal ordered), I think I'm not allowed to use a One-way ANOVA since this is based on means. A Chi-square test also seems not appropriate since it would ignore the ordinal nature of the 'Teacher' variable. So after reading some posts on this forum and also on some other websites I got the impression that a Kruskal-Wallis test might actually be appropriate.

Hopefully the situation is clear. Now for my question:
Is the Kruskal-Wallis (KW) test indeed the appropriate choice in my situation?

The reasons I'm doubting are:

  1. Many examples I see from the KW test show an ordinal variable with a lot more than 5 options.
  2. I read somewhere that the KW test is only appropriate for 'large' samples. They defined 'large' as >30, but is this per group or in total? In my case the total is 55, which should be fine, but per location the maximum is only 22.
  3. I also read that the KW test is not appropriate if you have many ties. I guess that with only a 5 point scale there might be too many ties. Should I check for this and if so what is the threshold? (perhaps this explains point 1).
  4. Are there any other conditions I need to check and if so how?

On a related issue, I'm using SPSS 19 and when I go for Analyze - Nonparametric tests - Independent samples', I can only enter scale variables in the fields list. Is there another solution besides having to change all my ordinal variables to scale? It's strange because in older versions of SPSS this was not a problem. [EDIT: Nevermind, I found the 'legacy dialogs' in the Nonparametric tests and can still use the old one to use the ordinal variables. I leave this in, in case someone has a similar problem]

Sorry for the long post, but hopefully it made my question(s) and situation clear. I'm looking forward to response and thanks in advance.

Kind regards,
Peter

p.s. English is not my native language. I've tried my best to avoid any spelling & grammar errors but might have overlooked some.
 
Last edited:

hlsmith

Less is more. Stay pure. Stay poor.
#2
Your post was very straightforward and easy to understand. I would recommend those procedures, if I was running your analyses. Sometimes these procedures have "exact" versions that are supposed to perform better with small samples.

Also, if you find significance, this will only mean that at least one of your groups differs. You will then need to perform pairwise comparisons of the groups using wilcoxon rank sum test. Lastly, it is typically prudent to adjust your level of significance when performing pairwise comparisons to account for chance of significance based on multiple tests. An easy and cautious correction many people use is the Bonferroni correction (multiply the determined p-values from multiple tests by the number of tests (e.g, p-value: 0.03 would then be 0.09 [if three comparisons] and would still be significant).
 
#3
Thanks hlsmith for your response. I take it then I can ignore my doubts?

Since I'm using SPSS a pairwise comparison is performed. SPSS uses the Dunn method, which takes the significance, multiplies it with the number of groups (k) (in my case 3), and then multiplies it by (k - 1)/2. In total this gives in my case alpha * 3 * (3 - 1)/2 = alpha * 3, so actually increasing the significance (so less likely to be significant). I'm not so worried about this though, since in most cases the adjusted significance still stays below the 0.05. Here's the link to the SPSS algorithm.

I've actually managed to mimic the SPSS output as shown here (click on the details to see the image) in Excel (I often do this to get a better understanding of the calculations), except for the significance of the pairwise sig. itself. I get the test statistic, std. error, std. test statistic and would also know how to get the Adj. sig. if only I could figure out how to get the sig. itself. SPSS mentions that it uses Z (see the first link), but NORMSDIST in Excel does not yield the same results.

This new sig. issue is a small issue. I'm already quite happy I can now at least report significant differences across the locations, rather than only a descriptive statistic persuading the reader.

Any further input (about my doubts) is of course still appreciated.