Measures of association

#1
Tried to use the search feature, but it appears to be broken.

Can a measure of association show a strong relationship while a test of significance shows that the results are not significant? If yes to either or both , can you provide some relevant examples illustrating how such situations might occur.

Trying to work on a project for school and this concept keeps stumping me.
 

hlsmith

Omega Contributor
#2
A scenario may occur if you have a small sample size that can't over come the possible variable given the sparse number of observations. So you have a large effect size but large variability opportunity.

This can be seen in say fisher exact tests and translated in effects via risk ratios. If I was at my computer I could kick out an example - perhaps in the morning.
 

spunky

Smelly poop man with doo doo pants.
#3
Example:

Code:
library(MASS)

dat1 <- mvrnorm(10, c(0,0), matrix(c(1,.5,.5,1),2,2))

             [,1]       [,2]
 [1,]  1.40927133  0.1398667
 [2,]  0.30122742  1.1474236
 [3,]  1.39751319  1.7424001
 [4,]  1.57680249 -0.1643728
 [5,]  1.31733439 -0.5057841
 [6,]  0.33860336 -0.4280187
 [7,]  2.07029519  3.1157457
 [8,] -0.08225975 -0.4649253
 [9,]  0.43302224  0.5914393
[10,]  1.15901794  2.0689712

> cor(dat1[,1],dat1[,2])
[1] 0.4928032
So r=0.493 is a pretty decent effect size. Actually, as per Cohen's definitions it would be considered a large effect size.

However, when you do the t-test for significance:

Code:
> cor.test(dat1[,1],dat1[,2])

        Pearson's product-moment correlation

data:  dat1[, 1] and dat1[, 2]
t = 1.6019, df = 8, p-value = 0.1479
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 -0.1983754  0.8566321
sample estimates:
      cor 
0.4928032
The p-value is 0.1479 which is not significant. Which is exactly what hlsmith said. You have a large population effect size of 0.5, it is estimated quite accurately to be 0.493 but the sample size is so small, N=10, that there's too much variability for it to cross the threshold of significance at .05
 
#4
Tried to use the search feature, but it appears to be broken.
Thank you for telling. It was supposed to be fixed "after the summer". But now we don't know if it is after the European summer or the Australian summer, or which year!

Imagine three numbers numbers like x = 1, 2 3.
What would y be to have a large correlation but no significance? Play around with it!

Edit: Spunky writes faster than I. (But 0.49 is not very high, is it? You must be able to do better than that. :) )
 

spunky

Smelly poop man with doo doo pants.
#5
But 0.49 is not very high, is it?
That depends on what frame of reference you're using for "high" or "low". In the social sciences we use Cohen's framework because in quite a variety of areas, a correlation of 0.5 is pretty high. But if you tried to extrapolate this to an engineering context, then I could see why a correlation of 0.5 is low. But that depends on the field and the fact that the OP does not define what "measure of association" or "strong relationship" mean makes trying to come up with a general answer very difficult.

I interpreted "measure of association" as the Pearson correlation coefficient and "strong relationship" the a Cohen's effect size of 0.5. Without clarifying these terms further the OP's question could be twisted around in a few ways, I think, so that the answer could be possible, impossible or a big "it depends".