Factor Analysis: Creating Reliable Scales

[DMCH]

Hello all.

I have a question I hope someone can help me with. I items which I have run PCA on and come up with 8 factors from about 20 vairables. I want to create a index to compare responses based on a classic political economic scale. i.e., some variables load heavily on items related to economic issues, and the rest on political issues. I constructed an index variable (i.e., index = v1+v2+v3.../Vn) from indications given by the PCA, however now I wonder if i should have just used the factor loadings. factor loadings are still confusing me at this point. Additionally. Factors have different N's for variables loading on them, and some are negatively loading on factors. Can someone help me rap my feeble head around this. Thanks yall.

DMCH

 

[noetsi]

I work with exploratory factor analysis (EFA) which in some respects is similar to PCA. I am not sure what you want to do. Are you trying to correlate the factors you created from PCA with other scales measuring economic or political concepts? Trying to see how various items loaded on the factors you created or something else? You need to clarify this for us to help you.

With EFA you would normally rotate the factors before you analyzed them because simple structure (which is critical to useful analysis) is generally only useful after rotation. I believe this is true in PCA as well (although again I don't work with that except as a tool for EFA). Generally you only look at items that load on factors above a certain level ( no one agrees what that should be, but in social sciences it would be rare to look at variables that loaded below the .3-.4 level on a factor).

Its the absolute value that determines this, that is a -.35 loading is no different in terms of whether it meets the minimum loading criteria than .35. In my experience, after rotation, strong negative loadings are rare, but that does not mean you won't find any. Obviously the sign changes the interpretation of the loading, I am not sure what a powerful negative loading means.

Rotation might well eliminate the strong negative loadings.

 

[DMCH]

@ Noetsi: Fair play, it was a bit unclear. So, what I have done is taken a survey i have put out there and gotten back and recoded all the data into (-) to (+) scores from lickert scales. not all the likert scale items are the same length, but this is because i have taken them from a survey already run in the US and was advised to leave good enough alone and not go around tinkering with tried and true surveys as a lowly grad student - as an aside. the question i really have now is this. so i have this giant wonderful survey back and i was like "yes! time to compare means!"....no. turns out my sample is heavily skewed, like my knowledge of statistics. again, another story.

so, i want to see how my sample compares the regional sample from the official survey. i ran a factor analysis before the deployment of the survey in the field to make sure the items were tapping (reasonably) attitudes on constructs of interest. I got the survey back and reran the FA with the sample alone and with the regional sample to see if they were still measuring the same constructs. they do, more or less the same. now armed with this confidence (personal, not necessarily statistical), i set to making index scores that i can use to plot into a multi-axis graph to show where my sample landed as compared to the regional sample. however a) i wondered if it didn't just make sense to save the factor scores as variables (probably regression or bartletts) and use those scores as the index scores to plot and present OR stay with my idea of building index scores by summing variable values for each respondent and dividing by Nvariables used. noted previously. i was also told that what i might consider was ranking the scores, finding the mean rank of the groups, and then subtracting the ranks from the mean and using these values as variables.

Now, i realize this is like my girlfriend talking to me about the difference in the old star trek movies compared to the new (bless her heart), but if anyone can help me come up with a sound way to (re)package my data and present it using factor scores OR indices, i would be really grateful. i hope that all makes sense now.

 

[spunky]

so, i want to see how my sample compares the regional sample from the official survey. i ran a factor analysis before the deployment of the survey in the field to make sure the items were tapping (reasonably) attitudes on constructs of interest. I got the survey back and reran the FA with the sample alone and with the regional sample to see if they were still measuring the same constructs. they do, more or less the same. now armed with this confidence (personal, not necessarily statistical), i set to making index scores that i can use to plot into a multi-axis graph to show where my sample landed as compared to the regional sample. however a) i wondered if it didn't just make sense to save the factor scores as variables (probably regression or bartletts) and use those scores as the index scores to plot and present OR stay with my idea of building index scores by summing variable values for each respondent and dividing by Nvariables used. noted previously. i was also told that what i might consider was ranking the scores, finding the mean rank of the groups, and then subtracting the ranks from the mean and using these values as variables.

to me it sounds like what you want to do here is multi-group confirmatory factor analysis where one group is the regional sample from the official survey and the other group is your sample.

comparing surveys (even if it's the same survey in different samples) without evidence of factorial invariance is pretty sketchy and has shown to either increase or decrease type 1 error rates (for if you were doing t-tests, for example) mask extra variability, etc.

but nothing is more sketchy than "factor scores". the fact that you can get drastically different solutions from different ways of getting them PLUS you can always rotate them to look like whatever you want leaves a pretty big door open for your methodology to be criticised.