Help with the level of measurement for correct/incorrect answers

[Billy the Poet]

Please, help bring in some lucidity and fill in any possible gaps in my reasoning.

The research assessed the participants' recall of a narrative story. One day after the presentation of the narrative, the participants were given the memory test. Their answers to some of the questions are then processed as correct, or 1, or incorrect answers, or 0. Those participants in the questionnaire who answered correctly deserve to be given scores whereas those who answered correctly get nothing Sort of like back in school when they gave out quizzes. These are clearly dichotomous data.

Now, I need to apply Stevens's typology and see which one of the four levels of measurement these data belong to. Usually, binary data are seen as nominal level. Examples of such are female/male, smoker/non-smoker, yes/no, etc. In these cases all the data would be equal and would differ only in the nominal characteristics. However, if the data are binary, but not equal and one value is greater than the other, it indicates the distance between the two values. Among scales measuring distance are ordinal, interval and ration. The last two are out as the data are, again, binary, have absolute 0, are discrete, and no information is given about how much one value is greater than the other.

This all points out to ordinal data that happens to be organized as dichotomous. In such case, the data can be compared and ranked. There is a clear difference between the values and it is also meaningful to the context of the analysis (correct answers vs incorrect). However, nothing can be said about how the difference itself. All what is certain is that 1 is superior to 0.

My question is, do you think there are any flaws in my reasoning? And, if no, do you know of any sources, such as books or papers, which mention the case of ordinal being dichotomous? This is important because I would like to see what others have written about it. One problem here is that there is not much what can be done with such data. If the data are ordinal, it means the only descriptive at disposal is a median, but that would result merely in either 1 or 0 of values, since the values are limited in range. I also have doubts about inferential tests. Maybe, there is another way around it.

 

[Karabiner]

You do not have 0 and 1. These are just labels, or codes. You have correct/incorrect.
You could as well code them 1 (correct) and 2 (incorrect), or 27 versus 100, or -1 versus +1, or anything else. Do not confuse the numbers you give as codes with numbers who mean to measure something (such as "recall ability = 0" versus "recall ability = 1.00" - which would not make much sense within your study, AFAICS).

With kind regards

K.

 

[Billy the Poet]

You do not have 0 and 1. These are just labels, or codes. You have correct/incorrect.
You could as well code them 1 (correct) and 2 (incorrect), or 27 versus 100, or -1 versus +1, or anything else. Do not confuse the numbers you give as codes with numbers who mean to measure something (such as "recall ability = 0" versus "recall ability = 1.00" - which would not make much sense within your study, AFAICS).

With kind regards

K.

Are you serious? I am sorry, but I think you either misunderstood my question or you did not even read it properly. Of course, I know these are labels. But, this is not my question.

 

[Karabiner]

All what is certain is that 1 is superior to 0.
My question is, do you think there are any flaws in my reasoning?

Yes. 1 is not superior to 0 in this case, both are not numbers.
You could have coded "correct" as 1 and "incorrect" as 2 without
making any mistake, or any other way. You have a categorial
(correct/incorrect) variable, not an ordinal variable.

Or maybe you want to count the number of correct answers for each participant,
in that case you have a count variable and can use procedures for ordinal
scaled variables.

With kind regards

K.

 

[GretaGarbo]

Are you serious? I am sorry, but I think you either misunderstood my question or you did not even read it properly. Of course, I know these are labels. But, this is not my question.

What is your question?

 

[hlsmith]

Yes, I am not following either. What is your question?

 

[Billy the Poet]

Yes. 1 is not superior to 0 in this case, both are not numbers.
You could have coded "correct" as 1 and "incorrect" as 2 without
making any mistake, or any other way. You have a categorial
(correct/incorrect) variable, not an ordinal variable.

Or maybe you want to count the number of correct answers for each participant,
in that case you have a count variable and can use procedures for ordinal
scaled variables.

With kind regards

K.

Thank you for reply!

Sigh. I know that 0 and 1 are not numbers. But, 0 is given to incorrect answers and 1 to correct. This context explains why 1 is superior to 0. When there is 1, it means something. When there is 0, it means nothing. This is the main reason why I would not classify the measurement of the variable as categorical.

Yes, exactly. The very reason I am using another scale is because 0 and 1 are not equal here. Of all the count data, I agree that the data are ordinal. But, I was wondering if you know of any material, in books or papers, where they discuss such situations in detail? In addition, do you know of any inferential tests I could use to analyze the data?

 

[Billy the Poet]

What is your question?

Yes, I am not following either. What is your question?

I apologize if the question was not made clear.

My question is, if the data are not nominal, is it right to go with ordinal instead?

In addition, do you know of any material in books or papers where dichotomous ordinal data are discussed? I am asking this since Google Books and Scholar did not yield any related results.

Also, if the data are indeed ordinal and dichotomous at the same time, what statistical tests will suffice to infer about how well participants in one group did on each question compared to the participants in another group? Sort of when we do independent t-test or ANOVA for ratio/interval. However, here is the case of dichotomous ordinal.

Does it make sense now?

 

[GretaGarbo]

Of course it it possible to infer something when the data are 0 and 1, like for example 'bad health' and 'good health' and when that is compared to 'exposed' and 'not exposed'. One example is a traditional chi-squared test in a contingency table. An other is to use a logit model where the '0 or 1' is the dependent variable.

 

[Billy the Poet]

Of course it it possible to infer something when the data are 0 and 1, like for example 'bad health' and 'good health' and when that is compared to 'exposed' and 'not exposed'. One example is a traditional chi-squared test in a contingency table. An other is to use a logit model where the '0 or 1' is the dependent variable.

But in such case are the data then nominal? This is important to establish before doing any tests on SPSS. If you had to put it in the context where the data are categorized as either Failed/Passed, or Absent/Present, what measurement would you use? Would Failed/Passed or Absent/Present be the same as, for example, Black/White or Male/Female?

 

[GretaGarbo]

I am not sure if Billy the Poet is mainly searching for confirmation that what he has already said is correct? If so, I would just say that it is fine! It is just fine! (But how would he react to other views?)

What do the other users here say?

We humans, and I in particular, are very good in searching for information that confirms the views we already have. But we find it harder to accept views that are in contrast to our present views. In a unified group there is a need for a devil's advocate.

 

[Billy the Poet]

I am not sure if Billy the Poet is mainly searching for confirmation that what he has already said is correct? If so, I would just say that it is fine! It is just fine! (But how would he react to other views?)

What do the other users here say?

We humans, and I in particular, are very good in searching for information that confirms the views we already have. But we find it harder to accept views that are in contrast to our present views. In a unified group there is a need for a devil's advocate.

I would not say I am looking for the views that necessarily coincide with mine. I genuinely find myself lost as both nominal and ordinal make sense and at the same time neither do. What I am looking for is to define the data.

This is an obvious pitfall of the logic Stevens and his supporters in typology were following when trying to organize psychometric data according to their scales. This was initiated in address to the bureaucratic matter. It is now omnipresent in psychometrics to such extent that to use a software for statistical analyses, one has to know which type the data belong to. However, there has been tons of criticism of the typology, most of which is legitimate. In the end, its authors were facing one the finest examples of the problems that Wittgenstein discussed in his work. They did not know how to categorize, so they went ahead and categorized in the most rigid and restricting manner which paid little attention to contexts. Ironically, it is contexts that play a role in most questions in social science research.

My main problem is that the data are clearly dichotomous. It makes sense to categorize it as nominal. But, the categories within the data are not nominal, as their difference is not due to their names but to their meaning. Clearly, correct is superior to incorrect. But, then again, the data cannot be ordered, because there are no other values to rank it with. In other words, having a correct answer in on observation automatically means it is not incorrect, and vice versa. This is why it could be still nominal.

But, then again, are data categorized by male/female really in the same measurement group as passed/failed in the context where one student gets a point and the other does not?

 

[Karabiner]

Male/female is ordinal, since male =1 and female = 0, and the variable measures maleneness, and each male gets a point but each female gets none.

Or maybe not.

Traditionally, and since you haven't made any reference to which construct you think is measured by the incorrect/correct variable on an ordinal scale, one will surely treat correct/incorrect like any other binary variable, i.e. use tests for binary-categorical variables.

Just my 2pence

P.

 

[Billy the Poet]

Male/female is ordinal, since male =1 and female = 0, and the variable measures maleneness, and each male gets a point but each female gets none.

Or maybe not.

Traditionally, and since you haven't made any reference to which construct you think is measured by the incorrect/correct variable on an ordinal scale, one will surely treat correct/incorrect like any other binary variable, i.e. use tests for binary-categorical variables.

Just my 2pence

P.

Yes, male/female can be ordinal, but only in the scenarios similar to what you described, and that is when a particular characteristic is addressed in the research. However, if a respondent is simply asked about the gender for demographic background, then the data will be binary.

So, the data I am inquiring about may be by definition ordinal, yet, because they are processed in two categories anyway, I should use the tests intended for the nominal data. Is this consistent with your suggestion?

Just to remind, the sample was divided into two groups. The memory test was given only once. The goal is to see if there are any statistically significant differences between the two groups on each question. Some questions seem to have ratio data, which makes the analysis parametric. However, other questions, such as those with dichotomous data, the situation might be more complicated and less familiar to me.

 

[Berley]

Answer this: why is a correct answer "superior" to an incorrect one? Sometimes it's better to get a wrong answer for the right reasons than a right answer for the wrong ones.

The wording of your original question demonstrates your strong belief that correct answers are better. (Correct answers "deserve" a reward and incorrect answers "get nothing of the sort" -- that is strong language.) That is a value judgment that, IMHO, has no place in your analysis. Value judgments are for the policy decision that gets made AFTER all the analysis is done. If you leave the value judgments out of it, then this is clearly nominal data.

Just my two cents.

 

[Billy the Poet]

Answer this: why is a correct answer "superior" to an incorrect one? Sometimes it's better to get a wrong answer for the right reasons than a right answer for the wrong ones.

The wording of your original question demonstrates your strong belief that correct answers are better. (Correct answers "deserve" a reward and incorrect answers "get nothing of the sort" -- that is strong language.) That is a value judgment that, IMHO, has no place in your analysis. Value judgments are for the policy decision that gets made AFTER all the analysis is done. If you leave the value judgments out of it, then this is clearly nominal data.

Just my two cents.

Berley, there are no value judgement or strong belief. The correct answer means the respondent could recall the information. The incorrect means the respondent failed to recall the information.

 

[GretaGarbo]

Clearly, correct is superior to incorrect.

There seemed (to me) to be a value judgement here.

Berley, there are no value judgement or strong belief. The correct answer means the respondent could recall the information. The incorrect means the respondent failed to recall the information.

But maybe not as much (value judgements) later on. Which (in my view) seems to be an improvement.

 

[Billy the Poet]

There seemed (to me) to be a value judgement here.




But maybe not as much (value judgements) later on. Which (in my view) seems to be an improvement.

In the former case, I intended to make the language as straightforward as possible to get to the point. So, we are on the same ground about the explanation. I think we can now return to the question. The goal is to see if there are any statistically significant differences between the two groups on each question.

If I understand Karabiner correctly, even if the data are ordinal, it has to be treated like binary (regardless if that meant ordinal or nominal). In the line with the aforementioned goals, what analyses or tests would you propose? I agree with your earlier notion of Pearson chi-sq. Mode would be an appropriate descriptive. Do you think logistic regression could help? That is considering that I have only one independent variable.