+ Reply to Thread
Page 2 of 2 FirstFirst 1 2
Results 16 to 27 of 27

Thread: differences between frequencies

  1. #16
    Pirate
    Points: 15,159, Level: 79
    Level completed: 62%, Points required for next Level: 191
    victorxstc's Avatar
    Posts
    875
    Thanks
    229
    Thanked 332 Times in 297 Posts

    Re: differences between frequencies




    Hi Thank you too
    Aha, maybe I have misunderstood it in the first place yes I too am interested in the answer.

    Thanks and Best
    "victor is the reviewer from hell" -Jake
    "victor is a machine! a publication machine!" -Vinux

  2. The Following User Says Thank You to victorxstc For This Useful Post:

    gianmarco (08-30-2012)

  3. #17
    TS Contributor
    Points: 40,089, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Downloads
    gianmarco's Avatar
    Location
    Italy
    Posts
    1,367
    Thanks
    232
    Thanked 301 Times in 225 Posts

    Re: differences between frequencies

    Dason...Trinker...and all the other smart guys, we need heeeeelp!!!
    http://cainarchaeology.weebly.com/

  4. #18
    Points: 2,657, Level: 31
    Level completed: 38%, Points required for next Level: 93

    Posts
    18
    Thanks
    5
    Thanked 0 Times in 0 Posts

    Re: differences between frequencies

    Thank you for the great discussion so far.

    It's basically about if subjects have chosen the same answers with the same frequency for phrasing A and phrasing B.

    So, if 6 choices are merged into 2 choices, I agree with victorxstc that we will loose data (in depending on how they are merged I think it will influence the result).

    However, gianmarco is also right about the dependent nature of the data.

    Btw: I have performed the chi-square test in SPSS in the meanwhile, which showed a difference for three of the five multiple choice questions.
    Last edited by poons; 08-31-2012 at 06:50 AM.

  5. #19
    Points: 2,657, Level: 31
    Level completed: 38%, Points required for next Level: 93

    Posts
    18
    Thanks
    5
    Thanked 0 Times in 0 Posts

    Re: differences between frequencies

    I have done some web searching the last week. In my understanding the chi-square test does not apply to dependent samples (as noted by gianmarco) so I tried to find an alternative to McNemar test for 2x6 contingency table but had no luck.

    Does anyone know if there is an alternative to McNemar if the data cannot be reduced to 2x2. I guess reducing the data to 2 categories would loose information in my case and the result would depend on how the answers of the multiple choice questions were merged.

    At some point victorxstc mentioned a Kolmogorov-Smirnov test. Would that still be a possible way to go?

    Thank you all for the great help so far!

  6. #20
    Pirate
    Points: 15,159, Level: 79
    Level completed: 62%, Points required for next Level: 191
    victorxstc's Avatar
    Posts
    875
    Thanks
    229
    Thanked 332 Times in 297 Posts

    Re: differences between frequencies

    In my understanding the chi-square test does not apply to dependent samples
    Are you sure? I think I have seen many studies in which one of the variables was a dependent one. But please clarify.

    Does anyone know if there is an alternative to McNemar if the data cannot be reduced to 2x2. I guess reducing the data to 2 categories would loose information in my case and the result would depend on how the answers of the multiple choice questions were merged.
    I haven't seen a non-2x2 alternative for McNemar.

    At some point victorxstc mentioned a Kolmogorov-Smirnov test. Would that still be a possible way to go?
    Maybe

  7. The Following User Says Thank You to victorxstc For This Useful Post:

    poons (09-14-2012)

  8. #21
    Omega Contributor
    Points: 38,289, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,992
    Thanks
    397
    Thanked 1,185 Times in 1,146 Posts

    Re: differences between frequencies

    Whew, I will admit that I just skimmed the last few posts since this was a long thread. It seemed like this eventually came up, but you need to keep in mind the subject level responses. What I am refering to is that you could have reported frequencies (10,20,30,40) than everyone read the second version and switched their responses but you once again got the same frequencies (10,20,30,40). However, individually each person changed their minds, making the question unreliable. This gets at my suggestions, perhaps an option is to look at multirater inter-rater reliabilities as well. Since in essence you are trying to compare responses between two things that are supposed to be the same.

  9. The Following User Says Thank You to hlsmith For This Useful Post:

    poons (09-14-2012)

  10. #22
    Points: 2,657, Level: 31
    Level completed: 38%, Points required for next Level: 93

    Posts
    18
    Thanks
    5
    Thanked 0 Times in 0 Posts

    Re: differences between frequencies

    Quote Originally Posted by victorxstc View Post
    Are you sure? I think I have seen many studies in which one of the variables was a dependent one. But please clarify.
    I found the following article about it and from what I understood chi-square should not be used.

    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1538510/

    which states that the chi-square requires "complete independence between rows and columns".

    @hlsmith:Could you please explain multirater inter-rater reliabilities a bit more. I googled a bit and from what I found out it compares ratings. However, I don't have a single rating but rather multiple selected answers which I think I can't merge into a single rating.

  11. #23
    Pirate
    Points: 15,159, Level: 79
    Level completed: 62%, Points required for next Level: 191
    victorxstc's Avatar
    Posts
    875
    Thanks
    229
    Thanked 332 Times in 297 Posts

    Re: differences between frequencies

    Quote Originally Posted by poons View Post
    I found the following article about it and from what I understood chi-square should not be used.

    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1538510/

    which states that the chi-square requires "complete independence between rows and columns".
    I'm confused then. Chi-square is used to assess the association between rows and columns. So if they are completely independent (zero association), using a chi-square test to assess their association is meaningless, in the first place. So this notion is self-contradictory. We are using a chi-square which needs zero association between rows and columns to assess the association between rows and columns that we are assuming to be zero?!

    So either there is something wrong with that paper, or I have understood here (or learned before) quite incorrectly.

  12. #24
    TS Contributor
    Points: 17,749, Level: 84
    Level completed: 80%, Points required for next Level: 101
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,540
    Thanks
    56
    Thanked 639 Times in 601 Posts

    Re: differences between frequencies

    It's basically about if subjects have chosen the same answers with the same frequency for phrasing A and phrasing B.
    For each answering alternative, perform a McNemar's test.

    Kind regards

    K.

  13. #25
    Points: 2,657, Level: 31
    Level completed: 38%, Points required for next Level: 93

    Posts
    18
    Thanks
    5
    Thanked 0 Times in 0 Posts

    Re: differences between frequencies

    Quote Originally Posted by Karabiner View Post
    For each answering alternative, perform a McNemar's test.
    Thank you, I thought of that. But how do I then decide if there is a significant difference or not? Is it a valid to say that if for all answers there is no significant difference then there is no significant difference for the whole question?

    Thank you!

  14. #26
    TS Contributor
    Points: 17,749, Level: 84
    Level completed: 80%, Points required for next Level: 101
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,540
    Thanks
    56
    Thanked 639 Times in 601 Posts

    Re: differences between frequencies

    Not necessarily, as far as I can see. But maybe one
    could make do with that anyway.

    Kind regards

    K.

  15. The Following User Says Thank You to Karabiner For This Useful Post:

    poons (09-14-2012)

  16. #27
    Points: 2,657, Level: 31
    Level completed: 38%, Points required for next Level: 93

    Posts
    18
    Thanks
    5
    Thanked 0 Times in 0 Posts

    Re: differences between frequencies


    @Karabiner: I have now performed a McNemar test for each answer for each question. For some answers there are significant differences for some not. For example, in one question their is no difference for 5 answers and a difference for 1 answer.

    I think that this should be ok, since I'm only interested if there is a difference in how often an answer has been selected. And I think their is no need to infer if their is a difference for the whole question. If their is no error in that reasoning I think I will go with the McNemar.

    @victorxstc: I'm not sure about this paper anymore ...

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats