URGENT: Missing Data, how to deal with it?

[Flo89]

Hi all,

I am currently having a problem with my data, and I have no idea what to do with it . I made a mistake while creating the online survey; I did not include one of the items in two of the scales (both was initially a 5-item scale), two of them are missing together because the statement are the same for both scales. After realizing the mistake, I added the item in halfway through the study, and now I have a massive amount of missing data.

Is there any way that I could deal with this problem?

Thanks in advance!

[victorxstc]

You mean that your scales were less first? Then I thin you can't fix that part. If you need a uniform sample, you have to convert the answers with more scales into those with less scales. Because a lower resolution cannot be converted to a higher resolution, unless with some degrees of guessing (which is data faking and fabrication). So you need to convert the high-resolution ones to low-resolution.

However if you can deal with a non-uniform sample, I suggest keeping the high-resolution answers, since they are valuable in their current format.

It is interesting that I was thinking of conducting an online survey study when I saw your post! Could you please tell me which system did you use (SurveyMonkey? etc.)? Did you first try other systems?

[hlsmith]

So clarify, you had the wrong response scale for some questions or did you have the same question twice? A little confused by your description.

[Flo89]

So sorry for not clarifying properly, what I meant was there was initially 5 questions for that scale, but I accidentally missed out one question. And it became 4 questions instead of 5 questions for that scale. After noticing the problem, I went on and add back that question in halfway through the study.
The reason why two questions are missing is because the statements for two of the scales are the same.
For example, participant will first rate the question 'how much do you know about who your teens contact using electronic devices'... and then beside that question, they will then have to rate their concern of that question.

[Flo89]

I am currently using Qualtrics to conduct my survey, I use that system because our school of psych provided us the account.

[hlsmith]

So you forgot to insert a question until half way through the survey. Well not much you can do there, perhaps impute the data, but I am not a fan of that. You are just out of luck. Report it in your results and then address in limitations. I once worked on a project where they accidently put the same question in twice, played it up as a way to test reliability, but reported that it was an error.

Still confused about the two questions. You used the wrong scale, or did you forget one questions, which had a follow-up question, so in essence you missed two???

[Flo89]

Yea, the latter one is correct. Because the questions were side by side:

E.g. How much you know about: Rate your concern:

nothing at all, very little, quite a lot, a great deal 1= not concerned 10=very concerned

Who your teen contact using electronic device? 1 2 3 4

Before dealing with it, I just want to confirm the nature of missing data... could this be considered as missing at random? or missing not at random?

Thanks!

[Lazar]

You can impute and this will be fine. The missing data is missing completely at random (experementor error) so there is no problem here (even listwise deletion would be unbiased here though you would have substantial less power).

Look here http://sites.stat.psu.edu/~jls/mifaq.html. For more detail look at the books by LIttle and Rubin or Craig Enders.

EDIT: The only thing maybe that would make the data missing at random is that there is a time or ordering effect here. As latter individuals got the question and earlier people did not. This is easily dealt with by creating a variable relating to the order (or date/time/whatever is appropriate) in which the participants completed the questionnaire. You then use this variable in your missing data model.

[Lazar]

[snip]Well not much you can do there, perhaps impute the data, but I am not a fan of that.[\snip]

@hlsmith: Why are you not a fan of imputation?

[Flo89]

You can impute and this will be fine. The missing data is missing completely at random (experementor error) so there is no problem here (even listwise deletion would be unbiased here though you would have substantial less power).

Look here http://sites.stat.psu.edu/~jls/mifaq.html. For more detail look at the books by LIttle and Rubin or Craig Enders.

EDIT: The only thing maybe that would make the data missing at random is that there is a time or ordering effect here. As latter individuals got the question and earlier people did not. This is easily dealt with by creating a variable relating to the order (or date/time/whatever is appropriate) in which the participants completed the questionnaire. You then use this variable in your missing data model.

Yea, I was considering listwise deletion, but I didn't want to lose that amount of data that I have collected. But is there a clear step-by-step way of conducting multiple imputation? I was only taught how to do Expectation maximisation in stats lecture. Would it be acceptable if I were to use Expectation Maximisation to deal with those missing datas? Thanks!

[Lazar]

EM is ok but it has no way of accounting for the uncertainty that comes from imputing data (it is no longer considered to be the best way to go). Instead you should use multiple imputations (5 data sets should be pleanty in this case). The website I gave will give you an idea. If you use SPSS multiple imputations can be done easily and can be found in the analysis drop down menu. Other options include the norm and amellia II packages, I also assume that SAS and STATA also do MI.

When it comes to analyising the data make sure you read the website I linked to carefully.

EDIT: All of this to say I think EM would be generally OK in this case BUT I would strongly consider learning how to do MI as it is a good skill to have.

[Flo89]

Ok will do. Thanks a lot for your help Lazar!