+ Reply to Thread
Results 1 to 11 of 11

Thread: Cramer's V effect size based on df

  1. #1
    Points: 2,355, Level: 29
    Level completed: 37%, Points required for next Level: 95

    Posts
    63
    Thanks
    25
    Thanked 0 Times in 0 Posts

    Cramer's V effect size based on df




    I saw a presentation about chi-square testing and correlations. Specifically calculating effect size for Cramer's V results. The puzzling thing was the according to the degrees of freedom. Basically higher df produces smaller values for the effect size to be large or medium.

    You can see what i mean in the presentation here:
    http://www.slideshare.net/edithosb/chi-square-in-spss

    Can anyone tell me why is that and how can i calculate what is the effect size(small,medium,high) for cramer's V when the df = 6?

  2. #2
    TS Contributor
    Points: 10,886, Level: 69
    Level completed: 9%, Points required for next Level: 364
    Karabiner's Avatar
    Location
    Schalke 04, Germany
    Posts
    1,677
    Thanks
    30
    Thanked 411 Times in 392 Posts

    Re: Cramer's V effect size based on df

    The relationship between w and Cramers V is: w = V * SquareRoot(r-1)
    where "r" is the smaller of the number of rows and number of columns
    (mind that in this context "df" does not mean degrees of freedom of
    the Chi square statistic). As you already know (from the presentation
    with r=2), w=0.1 may be assumed as "small", 0.3 as "medium", 0.5 as
    "large".

    Reference:
    Martin A Volker (2006): Reporting effect size estimates in school psychology research.
    Psychology in the Schools, Vol. 43(6), 653-672.
    http://www.ed.utah.edu/users/daniel....%282006%29.pdf

    Regards

    K.
    Last edited by Karabiner; 10-13-2011 at 03:37 AM. Reason: Removed a redundancy

  3. The Following User Says Thank You to Karabiner For This Useful Post:

    micdhack (10-12-2011)

  4. #3
    Points: 2,355, Level: 29
    Level completed: 37%, Points required for next Level: 95

    Posts
    63
    Thanks
    25
    Thanked 0 Times in 0 Posts

    Re: Cramer's V effect size based on df

    Karabiner thank you very much for the detailed response. I have two more questions based on what you wrote.

    Based on the above if i have a result such as V = 0.274, p = 0.007 with number of columns 3 and rows 4 then my r is going to be 3. Based on that my w will be 0.38 which translates into a medium effect size. Am i correct so far? Can i conclude that i have a correlation where one variable has a medium effect over an other variable?

    A second thing that i would like to ask is that i tried to test the correlation with Spearman's rho(ρ). The result was ρ = 0.288 and p < 0.01. How do i know what is the effect size for this? 0.288 is closer to 0 than above 0.5 but does this mean that i have to conclude that there is assosiation between the two variables?

    Regards,
    m.

  5. #4
    TS Contributor
    Points: 10,886, Level: 69
    Level completed: 9%, Points required for next Level: 364
    Karabiner's Avatar
    Location
    Schalke 04, Germany
    Posts
    1,677
    Thanks
    30
    Thanked 411 Times in 392 Posts

    Re: Cramer's V effect size based on df

    Mind that sample "effect sizes" are not effect sizes. Effect sizes refer to populations.
    Measures such as V or rho (or d or whatever), if they are calculated from sample data,
    are not real effect sizes, because they are composed of real effects and of chance variation.
    That's why we perform statistical tests -- sample "effects" could
    possibly be completely due to chance.

    Regarding your first question, I guess your result corresponds to what is described
    in the cited reference.

    Regarding your second question, I am not sure what you mean to ask. Whether there
    is an association between two variables is judged by the p-value, not by the size of
    the sample statistic (here: rho).

    Regards

    K.

  6. #5
    Points: 2,355, Level: 29
    Level completed: 37%, Points required for next Level: 95

    Posts
    63
    Thanks
    25
    Thanked 0 Times in 0 Posts

    Re: Cramer's V effect size based on df

    I thought that effect sizes could be applied to samples as well. Now i understand that this is not the case :S

    About the rho that's what i thought too! The p value if it is significant it means that there is a significant correlation at hand. But then i see websites such as this one (http://www.wikihow.com/Calculate-Spe...on-Coefficient) which basically say that the p tells you the chances of something occurring by chance (significance) but how strong is the correlation should be determined by the rho.

    And so in my example ρ = 0.288 and p < 0.01, the p tells me that i have a significant result but the rho is only 1/3 as strong as the ultimate correlation which would have been 1. Which poses the question, can i claim that i have a statistically significant correlation or not?

  7. #6
    TS Contributor
    Points: 22,734, Level: 93
    Level completed: 39%, Points required for next Level: 616
    Awards:
    Downloads
    gianmarco's Avatar
    Location
    Italy
    Posts
    1,085
    Thanks
    169
    Thanked 217 Times in 159 Posts

    Re: Cramer's V effect size based on df

    Hi,
    you have to pay attention to what any statistic is actually saying, and to what the p value is saying as well.
    They tell a different story.
    To keep with the correlation example, r informs us about the strenght of the correlation between the two variables at hand. You can have no correlation, a moderate or a strong or a perfect one.
    On the other hand, the p value is informing us about how likely is that the correlation you have found exists in the parent population. So, you can have a strong correlation BUT a non significant p value; or you can have a moderate or even weak correlation that turns up to be significant. This depends on the sample size: other things being equal, the bigger your sample, the more likely is that your correlation (even weak) truly exist in the population. If you have few data (i.e. smal sample) you simply do not have enough data to say if the correlation (even strong) truly exist in the population.

    Hope this helps
    Regards
    Gm
    Outside of a dog, a book is man's best friend. Inside of a dog it's too dark to read -Groucho Marx-

  8. The Following User Says Thank You to gianmarco For This Useful Post:

    micdhack (10-14-2011)

  9. #7
    TS Contributor
    Points: 10,886, Level: 69
    Level completed: 9%, Points required for next Level: 364
    Karabiner's Avatar
    Location
    Schalke 04, Germany
    Posts
    1,677
    Thanks
    30
    Thanked 411 Times in 392 Posts

    Re: Cramer's V effect size based on df

    And so in my example ρ = 0.288 and p < 0.01, the p tells me that i have a significant result but the rho is only 1/3 as strong as the ultimate correlation which would have been 1.
    A correlation r=0,3 is not so bad. Medium size. In those research areas which deal with living, non-technical systems you rarely will
    find correlations > 0.6 or so. Mind again, the size of rho in the population will be different (smaller or larger) from your sample rho.
    The p-value at least tells you that you can assume that in the population rho is NOT = 0.000000.

    Regards

    K.

  10. The Following User Says Thank You to Karabiner For This Useful Post:

    micdhack (10-14-2011)

  11. #8
    TS Contributor
    Points: 22,734, Level: 93
    Level completed: 39%, Points required for next Level: 616
    Awards:
    Downloads
    gianmarco's Avatar
    Location
    Italy
    Posts
    1,085
    Thanks
    169
    Thanked 217 Times in 159 Posts

    Re: Cramer's V effect size based on df

    Supplementing Karabiner's reply,
    I would add that, for both r or rho, it could be useful to provide the confidence interval, in order to get the idea of the variability of those statistics in the population.
    I believe that any statpack should routinely provide confidence interval for r or rho.
    A confidence interval could be "attached" with bootstrap procedures as well....But this is another story....

    Regards,
    Gm
    Outside of a dog, a book is man's best friend. Inside of a dog it's too dark to read -Groucho Marx-

  12. The Following User Says Thank You to gianmarco For This Useful Post:

    micdhack (10-14-2011)

  13. #9
    Points: 2,355, Level: 29
    Level completed: 37%, Points required for next Level: 95

    Posts
    63
    Thanks
    25
    Thanked 0 Times in 0 Posts

    Re: Cramer's V effect size based on df

    Awesome! Thanks to both of you! Btw, is there any easy way to calculate confidence intervals? SPSS doesn't have this feature for spearman or pearson

  14. #10
    TS Contributor
    Points: 22,734, Level: 93
    Level completed: 39%, Points required for next Level: 616
    Awards:
    Downloads
    gianmarco's Avatar
    Location
    Italy
    Posts
    1,085
    Thanks
    169
    Thanked 217 Times in 159 Posts

    Re: Cramer's V effect size based on df

    I do not recall if JMP provides this (I guess it does).
    You could try my Excel Template, where I implemented CI for Pearson'r. You find info in the "Other Software" section of this Forum.
    If you want a free tool for stat, you could consider to use the PAST program.

    If I recall in what program I saw CI for Spearman, I will let you know.

    Regards,
    Gm
    Outside of a dog, a book is man's best friend. Inside of a dog it's too dark to read -Groucho Marx-

  15. The Following User Says Thank You to gianmarco For This Useful Post:

    micdhack (10-15-2011)

  16. #11
    Points: 2,355, Level: 29
    Level completed: 37%, Points required for next Level: 95

    Posts
    63
    Thanks
    25
    Thanked 0 Times in 0 Posts

    Re: Cramer's V effect size based on df


    I have to say that i tried SAS, SPSS, Deducer for R and none of these programs had post hoc analysis for tests such as Friedman. The program that you suggested, PAST, it has right out of the box post hoc analysis for tests such as Friedman. In addition i tried spearman tests and its really easy. Basically as program is really practical.

    And btw it run on linux with wine.

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats