What is chance performance cut-off for a 2-part dichotomous response?

#1
If subjects had to give 2-alternative forced choice responses for a task (Y/N) and then rate the certainty of their response (also 2-alternative forced choice: High/Low), and we analyzed their responses in the 4 resulting bins (Y-Hi, Y-Lo, N-Hi, N-Lo), what would be considered "chance performance" -- less than 50% or 25%?
 

hlsmith

Omega Contributor
#2
I am always stepping out of my realm with probability. It depends how you frame the question. If the first split was random and the second was random the law of big numbers would say this converges toward a probability 25% of selecting any of the 4 groups.


That is treating everything independent and uniform. Probability of first group 50%, probably during second split 50%. 50% * 50% = 25%.


I could be wrong but this one seems straightforward given your description. So you could test if you distribution of outcomes was statistically different from the above, GIVEN your sample size.