Within Subjects Repeated Measures - Overall correlation across subjects

[TommyJimmy]

I’ll just start by giving a brief description of my study. It’s a memory study in which participants study an object and a cue stimulus together for each study trial. Whilst they study the stimuli I collect eye-tracking data to find out the number of fixations made towards the object and the total time spent looking at the object. In the test phase participants are presented with the cue and tested to see if they can recall the object that went with it. Each participant studies and is tested on 40 trials.

For each trial I therefore have these variables:

1. Number of fixations to object
2. Total time on object
3. Was the object later recalled yes or no?

I now want to know whether or not either number of fixations or time on target are correlated with subsequent recall. Whilst it seems simple enough to carry out this correlation within each participant’s data, I don’t know how to combine these correlations together into an across participants calculation.

I’m under the impression that, given the fact that there are both within- and between-participant sources of variance, I can’t simply put all of the data into one huge table and pretend that all the trials are from the same subject. One major problem I have in finding the correct way to perform this analysis is that I don’t even know a name to give to this problem. I’m sure I can’t be the only person to have had this problem but having asked around our department and searched through various forums I’m drawing a blank.

If you could help me out or point me in the right direction I’d really appreciate it.

[Junes]

It's definitely true that you shouldn't just put everything in one big table and run your analyses.

Your problem is in fact very common in (cognitive) psychology, although usually with the DV being some kind of continuous variable (for instance reaction time). There may be some differences because you measure something else, but this is how it's usually done with say reaction times:

You do two analyses: the F1 and the F2 (assuming you have F-tests, of course, otherwise you would call them differently). F1 is a test on the subject means. F2 is a test on the item means. The best case is for both of them to be significant. People used to combine them in a measure called minF, but that's not commonly done anymore I think. If one is significant but not the other, you're in a bit of a rut. Especially if the F1 is significant, you would probably still imply there's some kind of effect.

Now this is probably not the best way to do these kind of analyses. A better analysis would probably use multilevel modelling, as that's what I've seen the stats wizzes use. However, it's not very well-known in psychology circles and reviewers might not accept it.

[trinker]

I may not fully grasp your problem but have you considered regression anlaysis and simply dummy code your measured recall variable?

[TommyJimmy]

Hi I forgot to reply to this post after I wrote it. As an update I did try multi-level modelling. Bristol university have an online course about it which teaches you how to build a model using R, SAS or their own software. I chose R because it's free.

I included the analysis in my PhD thesis and I think it was just accepted because the examiners had never heard of it but it can't have been too wrong.