Testing hypothesis of difference in factor loadings across multiple scales in two questionnaires

I'm unsure of how to test this hypothesis and would greatly appreciate your ideas:
For illustrative purposes, let's assume that two groups of people (experts vs. novices) author a questionnaire with five scales (scale1 to scale5). Let's further assume that I have reason to hypothesize that when these two questionnaires are given to the same sample of respondents, factor loadings for the novice questionnaire would be equally high as the factor loadings in the expert version. I want to be able to accept or reject this hypothesis with a single test and not conduct multiple tests for each scale.
My initial intuition would be to do this using confirmatory factor analysis, but I'm not familiar enough with CFA's to know exactly how.
My questions:
  • Can I test this hypothesis by using CFA's and if so, how would I do it?
    • Do you have practical tips on how to do this in R lavaan ?
  • What other statistical frameworks and tests could be used?
  • If this is too difficult to pull off, do you have any other suggestions to how the two versions could be compared with regard to structural validity?
    • e.g. by averaging and comparing Cronbach's Alpha Coefficients (but what test would be used to do so)?
    • e.g. by comparing experts and novice facets with Generalizability Theory (comparing G-coefficients, but again, what test would be used?)
Thank you so much in advance.


Active Member
I had an opportunity to construct/analyze surveys before. I used the PCA factor loadings and Cronbach's to check for internal consistency. In my opinion, since PCA and the like are sort of exploratory I never bothered to test a formal hypothesis. I found it at least noteworthy to comment on the difference in factor loadings. Once you go beyond the first two components it gets difficult to interpret. Sorry my answer was kinda a non answer. Hope it helps in some way.

For me, the validity/consistency of the surveys were important but it was never my main question of interest. I was comparing the actual scores on the questionnaires. Perhaps that's another route you can go.
Last edited: