Confirmatory Factor Analysis: Appropriateness and interpretation

[shawn]

Greetings, all!

I am working on the statistics for my dissertation, and while it seems that confirmatory factor analysis fits my needs based on what I've read thus far, I am not quite sure how to run it and which software package I might need (I have heard good things about AMOS; my data is presently in SPSS).

Background info: There's an existing measure of cultural identity that a professor in my department developed. The measure has 55 items, each of which is endorsed on a Likert scale of 1-5. My professor administered the measure to some 3,000+ individuals. She performed an exploratory factor analysis on the resultant data and determined five factors from the associations between certain items. Subsequent statistical analyses indicated that the measure was reliable and valid.

Over 90% of the participants identified as White, making it difficult/inappropriate to generalize the results to people of color. My goal is to analyze the responses from the 300 non-White individuals to see if their response patterns result in the same five (or more or less) factors.

Again, I think that CFA is appropriate for what I want to do, but I'm not sure about:

1. How to run CFA on a software program (aiming for AMOS, most likely) and--

--this is the most critical part--

2. How to interpret the results of the CFA to determine whether the factors were identical and the extent to which they differ.

Thank you SO MUCH in advance for any help you can offer.

Cheers,
Shawn

[CowboyBear]

Hi Shawn,

It sounds like what you're after is testing measurement invariance. To do invariance testing, you might start by checking that the given factor model fits well in both samples individually. You could then perform multiple group analyses, where you sequentially apply constraints - starting with applying the same general factor model to both groups at the same time, then you might hold factor loadings to be equal across the groups, then both factor loadings and intercepts, and so on. This will give you an idea of the extent to which the factor model is invariant across the two groups.

A caution, though: a model with approximately 11 items per factor is inevitably going to do poorly on some CFA fit statistics, especially the model chi square (which tests a null hypothesis that the model fits exactly in the population(s) of interest). In effect the problem is that you're expecting a relatively simple model to do an awful lot (explain the covariances between a lot of items - 1540 covariance terms in all!) So it's good to start thinking about whether you're willing to accept a model that might fit the data "reasonably", but not closely.

One alternative to CFA could be to run EFA in both samples, but with a strong pre-selected criterion for how many factors to select. Parallel analysis or Velicer's MAP are better ways to determine the number of factors than the usual methods such as Kaiser's stopping rule or a scree plot. If the same number of factors is extracted in both samples, and the same items load on each factor, this supports the validity of the model across both samples. (This would be an unconventional way to go about dealing with this problem - just an idea).

1. How to run CFA on a software program (aiming for AMOS, most likely)

AMOS is linked closely to SPSS so does make some sense if it's freely available to you, but it's not the only choice. I really like the lavaan package in R. It's free, allows adjustments to fit statistics for breaches of normality (AMOS doesn't), and has a great measurement invariance function which automatically takes you through several steps of invariance testing (invariance testing in AMOS is quite time consuming). On the other hand AMOS allows you to specify models purely using a graphical user interface and path diagrams rather than code or syntax, which does make the learning curve a bit flatter.

As to how to actually run the analysis, the best thing is to look at the user's guide of the program you decide on and have a play with a program.

2. How to interpret the results of the CFA to determine whether the factors were identical and the extent to which they differ.

To help with this question and your analysis in general I'd suggest getting hold of Confirmatory Factor Analysis by Timothy Brown. As far as I know it's still the only book specifically focused on CFA, and is very readable and helpful.

[shawn]

Hello CowboyBear!

Thank you so much for your helpful response; per your suggestion, I have ordered Brown's Confirmatory factor analysis for applied research. I will also play around with Lavaan, as SPSS and Amos are available at my university, but I don't own licenses for either on my personal computer.

I failed to mention in my original question that while the measure has 55 items, they are separated into two scales (i.e., you can score high on Scale A and low on Scale B, low on A and high on B, low on both, or high on both; the five factors apply to each), and the maximum number of items that feed into a given factor on either scale is 9. Hopefully that won't be too much to overload the chi-square statistic, but I understand what you're saying regarding the fit of the models.

I also appreciate the alternative approach that you suggested regarding running an EFA on both. Moreover, as someone whose stats knowledge has eroded significantly since taking two semesters' worth during my first year of grad school, the latter technique somehow still makes sense to me.

Thanks again for your help!

Shawn