Calculating & Interpreting Cohen's D Effect Size

#1
Dear all, Im new to effect size & am struggling with some of the calculations, hence would like to seek some thoughts / advice.

1) Im calculating Cohen's Effect Size for 2 groups. Its not an intervention study, hence there's no control group per say. Im just comparing the measured characteristics of the 2 groups. Hence, in this case, does it matter which is the "experiemental" or "control group" ? Changing the order will determine if the effect size is positive or negative.

2) In the situation of a negative Cohen's Effect Size, how do i interpret it? Since <0.2 = trivial, does that mean all negative findings = trivial effect size then?

Thanks in advance !
 
#2
It doesn't matter which you choose to subtract from which, so long as you remember and acknowledge which order was chosen. In some cases there might be some precedent for using a certain group as the reference. Whether Cohen's D is negative or positive simply reflects the direction (which is interpreted based on what group you use as the reference), not the magnitude of the effect.
 
#3
It doesn't matter which you choose to subtract from which, so long as you remember and acknowledge which order was chosen. In some cases there might be some precedent for using a certain group as the reference. Whether Cohen's D is negative or positive simply reflects the direction (which is interpreted based on what group you use as the reference), not the magnitude of the effect.
Thanks for the reply; sorry im still a tad lose. So does that mean I ignore all the -ve signs n interpret the magnitude of the effect based on absolute figures? i.e. -1 I will read it as 1 which infers a large effect ?
 
#4
Thanks for the reply; sorry im still a tad lose. So does that mean I ignore all the -ve signs n interpret the magnitude of the effect based on absolute figures? i.e. -1 I will read it as 1 which infers a large effect ?
Yes, think of it asi it was an absolute departure from zero effect (from d=0) = |d|. Just to note, I have not seen negative "d" in any study. It has sense since d represents effect size of a factor (e.g. grouping, experimental). I know d is pretty standard, but I would preffer partial eta square (more intuitive interpretation of the statistics) or bayes factor.
 
#5
If that's the case, won't the confidence interval (CI) of the effect size (ES) be reversed too? I.e. for a -1.0 ES, the 90% CI is say -0.2 to 1.5. But since we view the -1.0 as an absolute 1.0, do we need to change the CI from -1.5 to 0.2 then ?
 

hlsmith

Omega Contributor
#6
Seems correct. I will point out, just in case you did not notice, your 90% CI includes 0 (AKA, no difference). Also the 90% CI is less cautious as the traditional 95% CI.
 
#7
ok here's some real figures from my calculations; will appreciate some enlightenment if my interpretation is correct. Basically im comparing the exercise hours of 2 groups of people; the level of significance is set at p<0.1 while the confidence interval (CI) for the cohen's d effect size is 90%.

P-value = 0.073, Cohen's d = 0.602, 90% CL for ES: -0.059 to 1.263

Can I conclude while there is a moderate effect size, it is unclear that this is a real effect as the CI range from a lower limit of trivial to an upper limit of large ?
 

hlsmith

Omega Contributor
#8
Hmmm. I would imagine that if your p-value is under 0.10 then your 90% CI should not include "0". Perhaps you only ran a one-sided test or you are calculating the two different ways? Or I am missing something.
 
#9
Hmmm. I would imagine that if your p-value is under 0.10 then your 90% CI should not include "0". Perhaps you only ran a one-sided test or you are calculating the two different ways? Or I am missing something.
Oh I ran an independent t-test for the 2 means. The 90% CI is not from the t-test but from Cohen's d effect size.
 

hlsmith

Omega Contributor
#10
I wouldn't report the p-value. Your effect size is not significantly different from a null value. Most likely based on the variability/sample size of your sample.
 
#11
Thanks to all who advised. Im still a little perplex why cant Cohen's d be negative? Let's explore this scenario 1: 2 independent sample Group A & B. This is an intervention study where A is the experimental group while B is the control.

Assuming we are testing a drug that helps to boost IQ & we are using an IQ test as the outcome measure. If Cohen's d is negative, can't we interpret it as the drug caused Group A to perform worse, hence its not working or even causing harm?

Situation 2 is where I'm having a hard time interpreting. Once again 2 independent sample Group A & B. But this time, its an observational study. There's no experimental nor control.

Assuming we have an IQ test with outcome as either pass or fail. Group A are those who pass & B those who fail. We compare qualities of both group such as hrs spend studying & hrs spend watching television. An independent T-test is run and found to be significant (P<0.05). We proceed with Cohen's d (Mean of A - B) and found a positive for hrs spend studying and negative for hrs spend watching television. How then do we interpret the negative in words ?
 

Karabiner

TS Contributor
#12
Is it so difficult to express that one group A has a higher mean than the other group, B?
And that this difference is expressed as + or -d?

By the way, calculations of d for a given sample are confused here with "effects".
Effects exist in the populations from which samples were drawn. The "d"
calculated within a sample is contaminated with sampling error, chance fluctuation.
So you aren't calculating an effect size for two groups (populations), but just an
effect size measure for two given samples, which is larger or smaller than the
true effect size.

With kind regards

K.
 

hlsmith

Omega Contributor
#13
Obviously it can be negative, just like odds ratios can be protective. But many times they are flipper improving understanding by a general audience. Present them any way you want.