G Theory / Variance Components problems

#1
I'm running SPSS Variance Components analyses using REML estimation. I have both real and fictitious data in which employees are rated by multiple raters. Subject ID and Rater ID are both treated as random.

On both real and fictitious data, the main effects seem to work ok. However, when I add the interaction between subject_ID and rater_ID to test for subject x rater effects, I get output where everything is .000 and a note says that the error component "is set to zero because it is redundant."

I followed up by running a univariate GLM which indicated the SS(error) and df for the interaction were both 0. I am assuming that may be related to the variance components errors, but I don't know how to work around the issue.

I've encountered this problem on a variety of different variables in both real and fictitious data. Any help would be greatly appreciated!
 

Dason

Ambassador to the humans
#4
In this new dataset is there more than one assessor?

It seems to me that you're just having difficulty conceptualizing which quantities are going to be estimable. You need to have replicates for the random components otherwise you can't estimate a variance.
 

Jake

Cookie Scientist
#5
Is each subject rated by a different group of raters, or is each subject rated by the same group of raters? If it is the latter, do the labels for "ID_Assessor" in the dataset correctly reflect this? (I.e., the same rater should always have the same label for ID_Assessor).
 
#6
You can make a cross table with raters and subjects. Do that cross table have at least two observations in each cell?

Does the title name refer to "generalizability theory"?
 
#9
You can make a cross table with raters and subjects. Do that cross table have at least two observations in each cell?

Does the title name refer to "generalizability theory"?

Thanks - this is a helpful suggestion. Yes, G Theory referred to Generalizability Theory.
 
#10
Darn, I'm still missing something. I've created a dataset in which 20 participants are rated by 3 raters, twice each (Excel version attached). So we have a Participant (20) x Rater (3) x Time (2) design, which I think should put enough observations in each cell, correct?

I also tried to manipulate the sources of variance (e.g., in one column, all variance is between-persons; in another, raters vary but are consistent across time, etc.). I've also added random error to make sure there is at least a little bit of variance on each facet. However, when I run the variance components analyses...I get all zeros again. :confused:

What am I missing? Thanks so much!