Confidence Interval in Statistics test

[jmvk]

Hi all,

I just had a test with the following question, of which I doubt the 'correct' answer:

Taking a level of significance of 5% in a hypothesis test and not rejecting the null hypothesis, is the same as saying:
A. We are 95% confident that the results have occurred by chance
B. We are 95% confident that the results have not occurred by chance
C. We are 5% confident that the results have not occurred by chance
D. None of the above

According to the answers, B is correct. How is this true? For example, if the p-value is 10%, how can we be 95% confident that the results have not occurred by chance?

 

[hlsmith]

D - they are all incorrect since you can't say we are ## confident. The convoluted definition is closer to, given repeated sampling of the population and correct model specification, 95% of confidence intervals will contain the true estimate. I think I got that wording right, but I am sure someone will correct me on nuances.

 

[jmvk]

D - they are all incorrect since you can't say we are ## confident. The convoluted definition is closer to, given repeated sampling of the population and correct model specification, 95% of confidence intervals will contain the true estimate. I think I got that wording right, but I am sure someone will correct me on nuances.

Thanks a lot!

 

[ondansetron]

D - they are all incorrect since you can't say we are ## confident. The convoluted definition is closer to, given repeated sampling of the population and correct model specification, 95% of confidence intervals will contain the true estimate. I think I got that wording right, but I am sure someone will correct me on nuances.

So, in the traditional sense of frequentist statistics, I think you can say "95% confident" because they didn't mean it as a probability statement on the interval. They meant it to refer to the methodology and long run success rate if used properly. Almost a short hand was of saying "this interval was generated by a method where 95% of all possible intervals contain the true parameter value and 5% don't, and this interval is X,Y". In other words, confidence has a different meaning than "probability" or "surety."

This discussion is aside from the question posted, because I think the question posted is a bad question.

 

[hlsmith]

If I was absolutely pressured to select between A-C, I would pick B. What they are probably trying to get at is that given alpha = 0.05, we are willing to accept results as extreme as revealed by chance 5% of the time, thus the opposite of that vague interpretation would be we are 95% sure the results are not by chance.

I think another component is whether this is for say a high school test or graduate level examine, not that it should ever matter, but the vague interpretation may help to deliver the general idea in a basic course, but the real concepts of significance testing based on p-values and confidence intervals is vary nuanced.