How to do data analysis for Likert scale questions?

#1
I have 8 survey questions (all Likert scale answers which I should give value of 1-5) that answer my research questions for each survey-taker. Which tests are best here to see if there is significant statistical difference across the 4 treatments and which treatment had the strongest effect??

I was thinking Kruskall-Wallis test and then a post-hoc test to see exactly which treatment had a sig difference, but am not sure how to conduct the entire analysis when I have 8 survey questions, do I need to do this bonferroni correction or anything? Any advice on how to approach the analysis and what to watch out for/consider with this many questions to evaluate would be much appreciated, Thanks!!
 
#3
Thank you @zstatman for providing the link. But I don't agree with them. I think it is silly statments made by them. (I think it is not worthy a university.) Although it is a very common statement.

For mee it is OK to compute a mean from a single Likert item. If they think that it is not OK, then, why is it suddenly OK to compute a mean from a sum of Likert items? It would still be a series of ordinal non-interval item. Summing them does not suddenly transform them to a ratio scales.

By the way, there are lots of clever people with lots of common sence that would go on and compute a mean from a Likert item. So, it is OK for mee.
(Maybe that university is doing that too when they sum their grade from high school.)
 

Miner

TS Contributor
#4
I agree with you Greta. I periodically analyze survey data with sample sizes in the thousands. Two questions may have distributions that are very different visually, yet have the same median and mode while the mean reflects the visual differences. While the mean may be somewhat biased in the theoretical sense, it is adequate and usable in a practical sense.
 

obh

Active Member
#5
It would still be a series of ordinal non-interval item
)
Hi @GretaGarbo,

We both agree that it okay to compute a mean from a single Likert item and I try to find the reason.
I think the reason is Psychology and not Statistics.

When I fill a Likert question I "feel" like it is interval scale...and you can't argue with my feeling :)

If most people feel like me it makes the Likert to be almost like an interval scale and this is the reason why average makes sense.

If for example, you would use the logarithmic scale:
How much do you like mushrooms soup?
1. 10
2. 100
3. 1,000
4. 10,000
5. 100,000

With this scale, would you use the average?
 
#6
For mee it is OK to compute a mean from a Likert item, especially if the the responder has seen the verbal items written together with the numbers 1, 2, 3, 4, 5.

But I as a user of the data, would not code it as 1, 10,...., 100 000. Then I would recode it as 1, 2, 3, 4, 5.
 

obh

Active Member
#7
But I as a user of the data, would not code it as 1, 10,...., 100 000. Then I would recode it as 1, 2, 3, 4, 5.
That was only A negative example :)
To prove my point that we do calculate the average of the Likert item because it psychologically appears to us an interval scale
 
#8
Thanks for the replies and thoughts, everyone!

Would it be ok to apply Kruskall Wallis test on each of the 8 Likert-scaled questions without other adjustments, or is the Bonferroni correction (so I guess dividing p-value by 8) necessary? The questions are measuring different things but I guess they are all coming from the same data set/answers. Would of course like to get significant results, would decreasing the number of questions (if it's possible) be recommended in this case?

I agree with you Greta. I periodically analyze survey data with sample sizes in the thousands. Two questions may have distributions that are very different visually, yet have the same median and mode while the mean reflects the visual differences. While the mean may be somewhat biased in the theoretical sense, it is adequate and usable in a practical sense.
When you analyzed the survey data, did you analyze each question on its own or did you have to adjust the data as well?
 

Karabiner

TS Contributor
#9
Is it very important to avoid type 1 errors? Or more important to avoid type 2 errors?
Or maybe you can just use a significance level of 1% instead of 5%.

With kind regards

Karabiner