+ Reply to Thread
Results 1 to 13 of 13

Thread: Calculating & Interpreting Cohen's D Effect Size

  1. #1
    Points: 23, Level: 1
    Level completed: 45%, Points required for next Level: 27

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Calculating & Interpreting Cohen's D Effect Size




    Dear all, Im new to effect size & am struggling with some of the calculations, hence would like to seek some thoughts / advice.

    1) Im calculating Cohen's Effect Size for 2 groups. Its not an intervention study, hence there's no control group per say. Im just comparing the measured characteristics of the 2 groups. Hence, in this case, does it matter which is the "experiemental" or "control group" ? Changing the order will determine if the effect size is positive or negative.

    2) In the situation of a negative Cohen's Effect Size, how do i interpret it? Since <0.2 = trivial, does that mean all negative findings = trivial effect size then?

    Thanks in advance !

  2. #2
    Points: 3,730, Level: 38
    Level completed: 54%, Points required for next Level: 70

    Posts
    155
    Thanks
    7
    Thanked 30 Times in 29 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size

    It doesn't matter which you choose to subtract from which, so long as you remember and acknowledge which order was chosen. In some cases there might be some precedent for using a certain group as the reference. Whether Cohen's D is negative or positive simply reflects the direction (which is interpreted based on what group you use as the reference), not the magnitude of the effect.

  3. #3
    Points: 23, Level: 1
    Level completed: 45%, Points required for next Level: 27

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size

    Quote Originally Posted by Disvengeance View Post
    It doesn't matter which you choose to subtract from which, so long as you remember and acknowledge which order was chosen. In some cases there might be some precedent for using a certain group as the reference. Whether Cohen's D is negative or positive simply reflects the direction (which is interpreted based on what group you use as the reference), not the magnitude of the effect.
    Thanks for the reply; sorry im still a tad lose. So does that mean I ignore all the -ve signs n interpret the magnitude of the effect based on absolute figures? i.e. -1 I will read it as 1 which infers a large effect ?

  4. #4
    Points: 4,939, Level: 44
    Level completed: 95%, Points required for next Level: 11
    Martin Marko's Avatar
    Posts
    53
    Thanks
    10
    Thanked 5 Times in 4 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size

    Quote Originally Posted by zeroX14 View Post
    Thanks for the reply; sorry im still a tad lose. So does that mean I ignore all the -ve signs n interpret the magnitude of the effect based on absolute figures? i.e. -1 I will read it as 1 which infers a large effect ?
    Yes, think of it asi it was an absolute departure from zero effect (from d=0) = |d|. Just to note, I have not seen negative "d" in any study. It has sense since d represents effect size of a factor (e.g. grouping, experimental). I know d is pretty standard, but I would preffer partial eta square (more intuitive interpretation of the statistics) or bayes factor.
    MM

  5. #5
    Points: 23, Level: 1
    Level completed: 45%, Points required for next Level: 27

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size

    If that's the case, won't the confidence interval (CI) of the effect size (ES) be reversed too? I.e. for a -1.0 ES, the 90% CI is say -0.2 to 1.5. But since we view the -1.0 as an absolute 1.0, do we need to change the CI from -1.5 to 0.2 then ?

  6. #6
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size

    Seems correct. I will point out, just in case you did not notice, your 90% CI includes 0 (AKA, no difference). Also the 90% CI is less cautious as the traditional 95% CI.
    Stop cowardice, ban guns!

  7. #7
    Points: 23, Level: 1
    Level completed: 45%, Points required for next Level: 27

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size

    ok here's some real figures from my calculations; will appreciate some enlightenment if my interpretation is correct. Basically im comparing the exercise hours of 2 groups of people; the level of significance is set at p<0.1 while the confidence interval (CI) for the cohen's d effect size is 90%.

    P-value = 0.073, Cohen's d = 0.602, 90% CL for ES: -0.059 to 1.263

    Can I conclude while there is a moderate effect size, it is unclear that this is a real effect as the CI range from a lower limit of trivial to an upper limit of large ?

  8. #8
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size

    Hmmm. I would imagine that if your p-value is under 0.10 then your 90% CI should not include "0". Perhaps you only ran a one-sided test or you are calculating the two different ways? Or I am missing something.
    Stop cowardice, ban guns!

  9. #9
    Points: 23, Level: 1
    Level completed: 45%, Points required for next Level: 27

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size

    Quote Originally Posted by hlsmith View Post
    Hmmm. I would imagine that if your p-value is under 0.10 then your 90% CI should not include "0". Perhaps you only ran a one-sided test or you are calculating the two different ways? Or I am missing something.
    Oh I ran an independent t-test for the 2 means. The 90% CI is not from the t-test but from Cohen's d effect size.

  10. #10
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size

    I wouldn't report the p-value. Your effect size is not significantly different from a null value. Most likely based on the variability/sample size of your sample.
    Stop cowardice, ban guns!

  11. #11
    Points: 23, Level: 1
    Level completed: 45%, Points required for next Level: 27

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size

    Thanks to all who advised. Im still a little perplex why cant Cohen's d be negative? Let's explore this scenario 1: 2 independent sample Group A & B. This is an intervention study where A is the experimental group while B is the control.

    Assuming we are testing a drug that helps to boost IQ & we are using an IQ test as the outcome measure. If Cohen's d is negative, can't we interpret it as the drug caused Group A to perform worse, hence its not working or even causing harm?

    Situation 2 is where I'm having a hard time interpreting. Once again 2 independent sample Group A & B. But this time, its an observational study. There's no experimental nor control.

    Assuming we have an IQ test with outcome as either pass or fail. Group A are those who pass & B those who fail. We compare qualities of both group such as hrs spend studying & hrs spend watching television. An independent T-test is run and found to be significant (P<0.05). We proceed with Cohen's d (Mean of A - B) and found a positive for hrs spend studying and negative for hrs spend watching television. How then do we interpret the negative in words ?

  12. #12
    TS Contributor
    Points: 17,779, Level: 84
    Level completed: 86%, Points required for next Level: 71
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,542
    Thanks
    56
    Thanked 640 Times in 602 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size

    Is it so difficult to express that one group A has a higher mean than the other group, B?
    And that this difference is expressed as + or -d?

    By the way, calculations of d for a given sample are confused here with "effects".
    Effects exist in the populations from which samples were drawn. The "d"
    calculated within a sample is contaminated with sampling error, chance fluctuation.
    So you aren't calculating an effect size for two groups (populations), but just an
    effect size measure for two given samples, which is larger or smaller than the
    true effect size.

    With kind regards

    K.

  13. #13
    Omega Contributor
    Points: 38,374, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: Calculating & Interpreting Cohen's D Effect Size


    Obviously it can be negative, just like odds ratios can be protective. But many times they are flipper improving understanding by a general audience. Present them any way you want.
    Stop cowardice, ban guns!

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats