A 6th response option ("I don't know") added to a 5-point Likert scale. Is data lost?

[streamline55]

Hello, everyone,

I need a little bit of help salvaging the data from a questionnaire.

One of my colleagues applied a questionnaire, but inadvertently, instead of using the original 5-point Likert scale (strongly disagree to strongly agree), he inserted a 6th answer into the scale. And, to make the matter worse, the 6th response option is … “I don’t know”.

The problem is the big proportion of respondents who, at one point or another, chose “I don’t know”. If they were a reasonably small percentage, I’d have just excluded them from the database.
However, the core of the research rests on a conceptual model, and excluding so many records would create a problem for the model.

Could someone point me in the right direction here? Are there any ‘good practices’, or can I do anything to use (transform, convert, etc.) those “I don’t know” responses?

Also, if I do any manipulation of the data in question (i.e., if I convert the “I don’t know” responses, by substitution, imputation, etc.), what kind of ‘disclaimer’, ‘warning’, annotation, should I use?

I know it is a long shot, but I confess, besides salvaging the responses, I am also curious what is the agreed practice (if there is one), in these type of cases.

Thank you !

PS: I know it sounds childish, but no, the ‘colleague’ isn’t me

 

[Englund]

Re: A 6th response option ("I don't know") added to a 5-point Likert scale. Is data l

The option 'I don't know' should be standard practice. If you do not include that option you're forcing respondents to choose an answer to a question which they don't know the answer to. The results are more reliable if you include the 'I don't know' option.

What would you gain if a lot of respondents answered questions they do not know the answer to? You'd get "nicer" results for publishing, but not better results.

 

[rogojel]

Re: A 6th response option ("I don't know") added to a 5-point Likert scale. Is data l

You might want to try a quick non-respondent analysis : if possible ask a few respondents why they answered with dont know? Is it because they genuinely didnt know or they wanted to avoid giving a definite answer?

regards

 

[streamline55]

Re: A 6th response option ("I don't know") added to a 5-point Likert scale. Is data l

The option 'I don't know' should be standard practice. If you do not include that option you're forcing respondents to choose an answer to a question which they don't know the answer to. The results are more reliable if you include the 'I don't know' option.

What would you gain if a lot of respondents answered questions they do not know the answer to? You'd get "nicer" results for publishing, but not better results.

Hi, Englund.
In psychological research, (validated) instruments are not be modified (never heard/read of such thing, unless it's adaptation of scales).
The IDK option does not offer any particular gain in information. I don't see any gain in reliability, also.
When a respondent doesn't know an answer, the usual instructions are to left it blank, unless there is a IDK option.

 

[streamline55]

Re: A 6th response option ("I don't know") added to a 5-point Likert scale. Is data l

You might want to try a quick non-respondent analysis : if possible ask a few respondents why they answered with dont know? Is it because they genuinely didnt know or they wanted to avoid giving a definite answer?

regards

HI, rogojel,
Thought of that - got answer as diverse as robbie williams' takes on life.
Some said "i've never encountered such situation", others "i though it would be the middle option", although it stood out that it wasn't (these were the most prevalent, most of the others were variants of these).
(the arrangement was strongly disagree - disagree - neither - agree - strongly agree - i don't know, where the 'i don't know' option was added by the interviewer for reasons i cannot comprehend...).
It is no surprise here, in terms of what we can speculate about people's reasoning when choosing IDK.
However, presuming without grounds, what they thought, would be risky and unsubstantiated (would not stand the critic) don't you think?

 

[rogojel]

Re: A 6th response option ("I don't know") added to a 5-point Likert scale. Is data l

hi,
this sounds like most who answered dont know would have picked the "neither" option, no? If one has njever seen a simlar situation the best answer would be to not chose the for or the against option - so their answer s consistent with "dont know" and "neither" being equivalent I think.

 

[CB]

Re: A 6th response option ("I don't know") added to a 5-point Likert scale. Is data l

I don't know of a standard solution here. As OP mentions, typically you would not include "I don't know" as an option to a Likert scale, because Likert items don't test knowledge in the first place - they guage agreement. More typically you would include a "Neither agree nor disagree" or "Neutral" option for the people who might not be familiar with the concepts in the stimulus statement, or who literally don't edge one way or the other.

I wonder about treating the "Don't know"s as missing data perhaps? Then use multiple imputation?

 

[streamline55]

Re: A 6th response option ("I don't know") added to a 5-point Likert scale. Is data l

I don't know of a standard solution here. As OP mentions, typically you would not include "I don't know" as an option to a Likert scale, because Likert items don't test knowledge in the first place - they guage agreement. More typically you would include a "Neither agree nor disagree" or "Neutral" option for the people who might not be familiar with the concepts in the stimulus statement, or who literally don't edge one way or the other.

I wonder about treating the "Don't know"s as missing data perhaps? Then use multiple imputation?

I agree. I was wondering about similar experiences and best/accepted practices, though.

 

[streamline55]

Re: A 6th response option ("I don't know") added to a 5-point Likert scale. Is data l

hi,
this sounds like most who answered dont know would have picked the "neither" option, no? If one has njever seen a simlar situation the best answer would be to not chose the for or the against option - so their answer s consistent with "dont know" and "neither" being equivalent I think.

I have strong doubts that ''DK" can be equated with "neither".
I've asked a selection of "DK" respondents what they thought at the time they selected DK and they said that they have never encountered that situation, others said that they couldn't picture the situation or didn't quite understand the wording, whereas, of course, a significant portion said they thought is the same with "neither".
However, there's no way to determine which thought what, and it would be presumptuous to assume "neither" instead of "DK"... I simply can't find justification for that.

 

[noetsi]

Re: A 6th response option ("I don't know") added to a 5-point Likert scale. Is data l

CWB I don't think don't knows are the same as missing data. In the latter case there is the presumption often that the respondent may not be answering the question deliberately, because they don't want to answer it. That is not true with don't knows, they simply are responding they are not familiar with the data. Of course that can be a dodge to avoid answering it...

One thing about don't know. Some make this as the middle value of the scale (so if there is a five point scale from strongly agree to strongly disagree it is the third, middle value). This is a really bad choice because there is no reason they are in the middle of the scale, they are saying they have no information or no view on the matter.

Regardless I avoid odd numbers of scales (say five points) because of the literature that says people automatically chose the middle point as a way of avoiding answering it. This then does not reflect a real view on the part of the respondent, they are not in the middle of the set of values in fact - its a way of avoiding answering the question. I always use an even set of scales, forced choice this is often called.