Can I calculate intrarater reliability using a single measure within a multiple subject system design? Can intrarater reliability be found without repeated measures on the same subject but on several different subjects??

My study is 30 physical therapists (blinded) evaluating the necks of 6 tests subjects once. I do have a gold standard to compare the therapists' findings against. The goal of the study was to first establish inter & intrarater reliability for administering the test, then I was going to look at validity if the reliability was acceptable.


If the answer is no, is it acceptable to go ahead with the assessment of validity if I only eval/find that the interrater reliability if good???

Thanks!

if you have 6 subjects, then technically you can do intrarater reliability for each therapist for those 6 subjects. If the subjects are all very similar in their diagnosis, then each therapist should have similar results for all of the patients. On the other hand, if the subjects are all very different from one another in their diagnosis, then you want the intrarater reliability to be low. It is a different type of a test then the more typical intrarater reliability that is done on the same patient repeatedly, but it would still accomplish your goal.