Hi,
I'm a vet doing a research project, but am not really strong in stats. I have a question about whether correction for multiple comparisons is needed in my study.
I am looking at 100 dogs (half with and half without disease). The disease is caused by a skull malformation. I have done CT exams (CAT scans) on all. I am measuring several areas of the skull to find any measurements that correlate with disease ( have done Student's t-tests). Prelim numbers show two (of about ten) bone measurements may be significant (P<<0.05).
Others in my dept think I need to correct for multiple comparisons since I have measured several structures. I cannot get my head around why I would need to in this particular case.
Each dog is either sick or not -- that makes two groups. Each subject already has each bone I measure (and a whole lot more I don't measure). At no point are these bone measurements compared to one another. Each is a fixed value independent of the others, and cannot change based on any other measurement. If the height of a bone correlates with disease status, but the width and length do not, how could my simply knowing those measurements weaken the significance of the height?
I seems that I should just ignore the eight insignificant measurements on my next 50 dogs, and guarantee the significance of the other two. But that seems crazy.
I do understand why, for example, a control group and three other groups who have been given drugs A, B, and C would need to be corrected, but those are different subjects with different treatments. But this is not my case.
I eventually have to defend these numbers so I need to understand them. Anyone have an opinion on whether or not I need to correct these values for a million comparisons? I'd love to hear it.
Thanks for your time and expertise.
JK
I'm a vet doing a research project, but am not really strong in stats. I have a question about whether correction for multiple comparisons is needed in my study.
I am looking at 100 dogs (half with and half without disease). The disease is caused by a skull malformation. I have done CT exams (CAT scans) on all. I am measuring several areas of the skull to find any measurements that correlate with disease ( have done Student's t-tests). Prelim numbers show two (of about ten) bone measurements may be significant (P<<0.05).
Others in my dept think I need to correct for multiple comparisons since I have measured several structures. I cannot get my head around why I would need to in this particular case.
Each dog is either sick or not -- that makes two groups. Each subject already has each bone I measure (and a whole lot more I don't measure). At no point are these bone measurements compared to one another. Each is a fixed value independent of the others, and cannot change based on any other measurement. If the height of a bone correlates with disease status, but the width and length do not, how could my simply knowing those measurements weaken the significance of the height?
I seems that I should just ignore the eight insignificant measurements on my next 50 dogs, and guarantee the significance of the other two. But that seems crazy.
I do understand why, for example, a control group and three other groups who have been given drugs A, B, and C would need to be corrected, but those are different subjects with different treatments. But this is not my case.
I eventually have to defend these numbers so I need to understand them. Anyone have an opinion on whether or not I need to correct these values for a million comparisons? I'd love to hear it.
Thanks for your time and expertise.
JK