Chi square effect size


New Member

I have the following data:
# successs # non success
population1 12 2293
population2 82 4926

I ran chi square test on the table above using the tool: I got the following results: chi square = 14.65 and p-value =0.0001. I interpreted this results as the null hypothesis is strongly rejected and that the difference in population does have a an effect on the success rate (I am not sure if this is the right interpretation). The problem is that the Cramer's V is Nan. I calculated the phi value (chi square / n) and I got a very small number which means that the correlation is not significant (0.002). I am confused by this result because I think the pattern is clear as the probability of success in population one is 3.2 times the probability of success in population two. I searched the web and saw that chi square is not good beyond a data size or that I should use Fisher exact test which I tried without success.

Another point, how can we be so sure about rejecting the null hypothesis and then get a weak association suggesting that there is no correlation.

Your help is highly appreciated.


Omega Contributor
Fisher exact test which I tried without success.

I could flip a coin a kabillion times and get 0.0001% heads, then flip another coin and get 0.0003% heads. Well I can conclude that the rates are different and hey they might be as the rate converges to the truth with every trial. Is that a big effect. Nah. But the study is well powered to find a miniscule effect size.


New Member
Thank you for your reply. The phenomenon I am observing is rare in both populations by nature. So I am not surprised by the small percentages. What I would like to say is, despite this phenomenon is rare (in both populations), it is more frequent/likely in one of them (again the probability ratio is 3.2 times). Is there a way to say that or I am wrong?

Another way to put my question, can I only report p value with the probability? Would that be statistically acceptable in this exceptional situation as the effect size of the phenomenon I am studying is small by nature.

Thanks again