+ Reply to Thread
Page 1 of 4 1 2 3 4 LastLast
Results 1 to 15 of 48

Thread: Likert Scale (1-5) statistical analysis

  1. #1
    Points: 78, Level: 1
    Level completed: 56%, Points required for next Level: 22

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Likert Scale (1-5) statistical analysis




    Hi, I have done my search on this website and google and have found my statistical test to be either t-test or chi-square so I'd like you guys to tell me which one is more valid since I am getting conflicting information. I don't want to argue if this is ordinal or interval data as even the academics bicker over this and I just want to find the right statistical test that is valid and will tell me what I want to find.

    Background: I am doing a group research project. We are assessing the preference of product A to Product B in a certain specific population. We set up 8 questions in Likert Scale from 1-5. It is important to note that these 8 questions is actually 4 questions but split up. What I mean is that, for example, Questions 1: "Do you think product A is effective at doing what it does" Questions 2: "Do you think product B is effective at doing what it does" and that will be one group. Then group 2 would be questions 3-4 asking "do you prefer to use product A as your top choice" and "do you prefer to use product B as your top choice." I think you guys get what I am trying to get at.

    Thank you for your help, our group has been stuck at this for sometime now and I even contacted our school's biostatistician and he said we should do chi-square but what do you guys think? We also plan to give descriptive statistics as well.

  2. #2
    TS Contributor
    Points: 22,399, Level: 93
    Level completed: 5%, Points required for next Level: 951
    spunky's Avatar
    Location
    vancouver, canada
    Posts
    2,135
    Thanks
    166
    Thanked 537 Times in 431 Posts

    Re: Likert Scale (1-5) statistical analysis

    i think it would also help us out to know what your research question is, i.e. what are you looking for. for instance, do you want to see which product is preferred? are you intersted in seeing whether the wording of the questions generate different answers? both? there are several ways to go here depending on what you're looking for...
    for all your psychometric needs! https://psychometroscar.wordpress.com/about/

  3. #3
    Points: 78, Level: 1
    Level completed: 56%, Points required for next Level: 22

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Likert Scale (1-5) statistical analysis

    We are trying to see whether this subgroup population prefer product A or product B.

    The questions are in the following group
    1-2: you prefer product a or product b as your number one choice for doing this thing
    3-4: I think product a work better than product b and verse visa (yeh this is a poorly worded question)
    5-6: I think product a or product b works in doing this thing
    7-8: I think product a or product b has no bad adverse events.

    so as you can see, its basically trying to assess whether these people prefer product a or b based from these questions. We also collected demographic data such as education, sex, age, occupation. So I was think a chi square would work but I been reading that a t-test would also work?

  4. #4
    TS Contributor
    Points: 6,786, Level: 54
    Level completed: 18%, Points required for next Level: 164

    Location
    Sweden
    Posts
    524
    Thanks
    44
    Thanked 112 Times in 100 Posts

    Re: Likert Scale (1-5) statistical analysis

    So I was think a chi square would work but I been reading that a t-test would also work?
    A t-test "works". But the results will be flawed. Since data on likert scale usually have smaller variance then "real" data, the p-values you get will be smaller. Thus you'd get a higher chance of getting a positive result, even though there aren't any. F-test on the other hand, is possible to use.
    Gene Glass’ famous Monte Carlo study of ANOVA in which Glass showed that the F-test was incredibly robust to violations of the interval data assumption (as well as moderate skewing) and could be used to do statistical tests at the scale and subscale (4 to 8 items but preferably closer to 8) level of the data that was collected using a 5 to 7 point Likert response format with no resulting bias
    [29]
    Conclusion: A t-test on likert scale data will probably result in a higher chance of getting a type I error. An F-test though, is possible to use during some conditions.

    Carifio, J., & Perla, J, R. (2007). Ten Common Misunderstandings, Misconceptions, Persistent Myths and Urban Legends about Likert Scales and Likert Response Formats and their Antidotes. Journal of Social Sciences, 3, 106-116.

  5. #5
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: Likert Scale (1-5) statistical analysis

    Quote Originally Posted by Englund View Post
    A t-test "works". But the results will be flawed. Since data on likert scale usually have smaller variance then "real" data, the p-values you get will be smaller. Thus you'd get a higher chance of getting a positive result, even though there aren't any. F-test on the other hand, is possible to use.Conclusion: A t-test on likert scale data will probably result in a higher chance of getting a type I error. An F-test though, is possible to use during some conditions.
    Hi there,

    A couple of things confuse me a bit here:

    Can you explain a little more what you mean by "real data"?

    How can an F-test be ok but t-test not? t and F are very closely related - if a variable X is distributed as t with df=m, then X^2 is distributed as F with df1 = 1 and df2 = m. So in an applied sense if there are just two groups and one DV, t-test and F-test will provide exactly the same p value...

  6. The Following User Says Thank You to CowboyBear For This Useful Post:

    Dason (07-15-2012)

  7. #6
    TS Contributor
    Points: 22,399, Level: 93
    Level completed: 5%, Points required for next Level: 951
    spunky's Avatar
    Location
    vancouver, canada
    Posts
    2,135
    Thanks
    166
    Thanked 537 Times in 431 Posts

    Re: Likert Scale (1-5) statistical analysis

    i was gonna follow-up on this post but i just realized that the question postedby CBear to Englund is considerably more interesting, so i'll wait & see...
    for all your psychometric needs! https://psychometroscar.wordpress.com/about/

  8. #7
    TS Contributor
    Points: 6,786, Level: 54
    Level completed: 18%, Points required for next Level: 164

    Location
    Sweden
    Posts
    524
    Thanks
    44
    Thanked 112 Times in 100 Posts

    Re: Likert Scale (1-5) statistical analysis

    Quote Originally Posted by CowboyBear View Post
    Can you explain a little more what you mean by "real data"?
    Simply what I meant: People do not answer correctly since they are affected of different kinds of cognitive bias et cetera. Further, the "real" or "true" values are unknown in most cases.
    Quote Originally Posted by CowboyBear
    How can an F-test be ok but t-test not? t and F are very closely related - if a variable X is distributed as t with df=m, then X^2 is distributed as F with df1 = 1 and df2 = m. So in an applied sense if there are just two groups and one DV, t-test and F-test will provide exactly the same p value...
    That is true. But if you look at what I cited in my last post you'll partly get your answer. See below.
    F-test was incredibly robust to violations of the interval data assumption (as well as moderate skewing) and could be used to do statistical tests at the scale and subscale (4 to 8 items but preferably closer to 8) level of the data that was collected using a 5 to 7 point Likert
    So, an F-test where there are only two groups and one DV, will also be flawed.

  9. #8
    TS Contributor
    Points: 6,786, Level: 54
    Level completed: 18%, Points required for next Level: 164

    Location
    Sweden
    Posts
    524
    Thanks
    44
    Thanked 112 Times in 100 Posts

    Re: Likert Scale (1-5) statistical analysis

    It is possible that I'm speaking out of my night cap*. I am simply saying what I've come to understand from the article posted earlier in this thread.

    *a phrase directly translated from Swedish and means that it is possible that I don't know what I am talking about :P

  10. #9
    Points: 5,259, Level: 46
    Level completed: 55%, Points required for next Level: 91
    SiBorg's Avatar
    Posts
    255
    Thanks
    71
    Thanked 25 Times in 22 Posts

    Re: Likert Scale (1-5) statistical analysis

    Why not use the Mann-Whitney U test and therefore only worry about ranking the data rather than the underlying distribution. It's possible to rank Likert data but not so easy to determine whether it is truly a continuous scale (which it probably isn't).

    That's what I've used when analysing Likert data.

  11. #10
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: Likert Scale (1-5) statistical analysis

    Personally I think its reasonable to assume that scales that involve Most satisified to Least satisfied are continuous, but there is broad disagreement on this issue.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  12. #11
    TS Contributor
    Points: 6,786, Level: 54
    Level completed: 18%, Points required for next Level: 164

    Location
    Sweden
    Posts
    524
    Thanks
    44
    Thanked 112 Times in 100 Posts

    Re: Likert Scale (1-5) statistical analysis

    Quote Originally Posted by noetsi View Post
    Personally I think its reasonable to assume that scales that involve Most satisified to Least satisfied are continuous, but there is broad disagreement on this issue.
    That is most often wrong. In most cases, the difference between 4 and 5 (on a five level scale) is bigger than the difference between 2 and 3 or between 3 and 4. That is also why a simple t-test gives flawed results and why it has to be seen as ordinal data. Please correct me if I'm wrong.

  13. #12
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: Likert Scale (1-5) statistical analysis

    Quote Originally Posted by Englund View Post
    But if you look at what I cited in my last post you'll partly get your answer. See below.So, an F-test where there are only two groups and one DV, will also be flawed.
    I just don't quite see how you're drawing the conclusion that a t-test will "work" with Likert data but not an F-test... as far as I can tell, Carifio & Perla don't discuss t-tests in that article, just F-tests. (I haven't fully read the article though - the writing style is so grating that I had to look away).

    The accuracy of their claim about the F-test is pretty dodgy to start with though. Carifio and Perla claim that: "Gene Glass’ famous Monte Carlo study of ANOVA... showed that the F-test was incredibly robust to violations of the interval data assumption."

    This is nonsense. Glass et al's study was concerned with distributional assumptions like homoscedasticity and error normality. It doesn't even mention the issue of measurement levels (ordinal, interval, etc). The assumption of interval measurement concerns the meaning of data points and intervals on the particular scale, not whether the data is continuous or normally distributed.

    Actually, it's sort of an interesting question whether it's possible or reasonable at all to use simulation studies to check the robustness of statistical tests to the use of ordinal data. I don't think we can really do this unless we make quite specific assumptions in turn (e.g. like assuming that an ordinal measurement scale is formed by dividing a latent normal distribution into a discrete set of response categories). Spunky?

  14. #13
    TS Contributor
    Points: 22,399, Level: 93
    Level completed: 5%, Points required for next Level: 951
    spunky's Avatar
    Location
    vancouver, canada
    Posts
    2,135
    Thanks
    166
    Thanked 537 Times in 431 Posts

    Re: Likert Scale (1-5) statistical analysis

    ok, so here's spunky's input into the issue:

    Carifio & Perla's article is probably as boring as things get. there's just blah, blah, blah on why Likert-type scales do this or that. very little of it is worthwhile to remember... IMHO, the only useful thing here is the citation for the Glass et. al. simulation. now, with that being said, Glass' article is pretty limited when it comes to this issue of the impact of the scale of measurement on the properties of statistical tests. and why is that? well... because they only consider ONE-WAY ANOVA designs. it doesnt take a statistician to deduce that the more complex a technique gets, the more and more it'll rely on its assumptions, and in the ANOVA context this happens as soon as factorial designs take centre stage. scales of measurement can screw your ability to detect interactions, and dont even get me started with ranked data in repeated-measures ANOVA.

    but aaaaaaaaaaaaaaaaaaaaanyway... the issue at hand is the t-test, right... well... here it goes.

    BEHOLD PEOPLE! THE SIMULATION TO END ALL SIMULATIONS... CAN T-TESTS BE USED IN ORDINAL DATA?

    Code: 
    ##SIMULATION PARAMETERS
    set.seed(1)
    reps <- 1000 #numebr of repetitions for the simulation
    rezu <- double(reps) #vector to store the results of such simulation
    n <- 30 #sample size
    
    ### ACTUAL SIMULATION##
    
    for (i in 1:reps) {
    
    X1 <- rnorm(n)
    X2 <- rnorm(n)
    U1 <- as.numeric(cut(X1, breaks=c(-Inf, mean(X1)-sd(X1), mean(X1), mean(X1)+sd(X1), Inf), labels=c(1:4)))
    U2 <- as.numeric(cut(X2, breaks=c(-Inf, mean(X2)-sd(X2), mean(X2), mean(X2)+sd(X2), Inf), labels=c(1:4)))
    
    rezu[i] <- t.test(U1,U2)$p.value
    }
    
    summary(rezu)
    sd(rezu)
    plot(density(rezu))
    these are the results i get:

    Code: 
    > summary(rezu)
       Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
     0.2784  0.6919  0.7985  0.8016  0.8950  1.0000 
    > sd(rezu)
    [1] 0.1509686
    conclusion? YES, YOU CAN USE ORDINAL DATA FOR A T-TEST

    ok, ok... i hope people notice that i'm just being a little facetious when it comes to this issue. there are many things i'm assuming here: (equally-spaced intervals of discretization, normal distributions, etc...) but i think that code is simple and flexible enough for other people to expand on it. for the time being the fact that it preserves type 1 error is enough to believe that, at least in the case of 4 likert points, we can consider ordinal data as continuous. the more points you use, the better the estimate, of course.

    so to the OP, you should be alright... i think. lol
    for all your psychometric needs! https://psychometroscar.wordpress.com/about/

  15. The Following 2 Users Say Thank You to spunky For This Useful Post:

    CowboyBear (07-16-2012), trinker (07-16-2012)

  16. #14
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: Likert Scale (1-5) statistical analysis

    Quote Originally Posted by spunky View Post
    so to the OP, you should be alright... i think. lol
    First time a statistical simulation has been reported where the concluding word is "lol"?

  17. #15
    TS Contributor
    Points: 22,399, Level: 93
    Level completed: 5%, Points required for next Level: 951
    spunky's Avatar
    Location
    vancouver, canada
    Posts
    2,135
    Thanks
    166
    Thanked 537 Times in 431 Posts

    Re: Likert Scale (1-5) statistical analysis


    Quote Originally Posted by CowboyBear View Post
    First time a statistical simulation has been reported where the concluding word is "lol"?
    pfff.... you should look at my manuscripts!

    it's an editing nightmare for my advisor ha!
    for all your psychometric needs! https://psychometroscar.wordpress.com/about/

+ Reply to Thread
Page 1 of 4 1 2 3 4 LastLast

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats