+ Reply to Thread
Results 1 to 14 of 14

Thread: Multiple Group Analysis necessary?

  1. #1
    Points: 1,587, Level: 22
    Level completed: 87%, Points required for next Level: 13

    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Multiple Group Analysis necessary?




    Hi,

    for my master's thesis I collected data with an online survey, measuring consumer behaviour. I used two samples, one group from Australia and one group from Malaysia. I also created a model that is supposed to measure this behaviour, but now I am wondering whether I should apply data from both samples combined to test the model, or separate the two samples and perform two separate tests. The method I am using is structural equation modeling.
    Combining the data is much easier, but I'm not sure if it is the right way.
    I also want to compare the groups on their mean scores. Is that possible with SEM (Amos) or should I just perform a t-test?

    Thanks in advance for your answer!


    Robert

  2. #2
    Points: 1,587, Level: 22
    Level completed: 87%, Points required for next Level: 13

    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts
    After finding some tutorials, I tried to run some more tests with the data and Amos, but I keep having troubles with comparing my data using multiple group analysis. Is it really a necessary procedure to separate the two samples and compare the two corresponding models, even if the model using both samples has better fit and makes more sense?
    I don't want to complicate things for no reason.

  3. #3
    Phineas Packard
    Points: 16,013, Level: 81
    Level completed: 33%, Points required for next Level: 337
    Lazar's Avatar
    Location
    Sydney
    Posts
    1,159
    Thanks
    198
    Thanked 336 Times in 299 Posts
    Quote Originally Posted by robertz View Post
    After finding some tutorials, I tried to run some more tests with the data and Amos, but I keep having troubles with comparing my data using multiple group analysis. Is it really a necessary procedure to separate the two samples and compare the two corresponding models, even if the model using both samples has better fit and makes more sense?
    I don't want to complicate things for no reason.
    I am not sure how AMOS works never having used it but Byrne has an introduction where she goes into detail on how to fit a multi-group model in AMOS.

    Is it necessary? I think it is for several reasons yes it is:
    1. The measurement structure underlying your model may be different across cultures which calls into question any results you might get from your structural parameters and mean-difference tests (see an old ****ty chapter of mine for examples. Parker, P., Dowson, M., & Mcinerney, D. (2007). Standards for Quantitative Research in Diverse Sociocultural Contexts. In D. McInerney, S. Van Etten, & M. Dowson, Standards in Education (pp. 315-330). Charlotte: Information Age Press.).

    2. Surely you are interested as to whether culture makes a difference. Multigroup analysis allows you an easy way of determining whether parameter estimates are statistically significantly different for different cultures.

    3. Multi-group analysis is relatively easy once you know how to fit your model and I think is conceptually easier to report than other options. It is a worthwhile tool to have in your kit.

    Two final notes:
    1. With multigroup analysis you are NOT actually fitting separate models for separate groups. Rather you run a model in which the parameters you are interested in are constrained to be equal across groups and then compare the fit of that model to one in which the parameters are free to vary across groups. If, by way of a chi-squared difference test, the models are not significantly different then your model is equivalent across cultures. If they are not then at least one of the parameters is different between your two cultures. I think Byrne outlines how to do this in detail. If not post back here and I will run you though the logic.

    2. In relation to mean differences, yes I would just use a series of t-tests if they are not the main focus of your study.

    Hope this helps.

  4. #4
    Points: 1,587, Level: 22
    Level completed: 87%, Points required for next Level: 13

    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks for your help, Lazar. I managed to get a copy of the book by Byrne, and it is very useful. I'm still very new to the whole process of SEM, so I still have some basic questions, if you don't mind.
    To come up with properly measured and reliable variables, I conducted factor analysis and computed cronbach's alpha, using the data of both samples combined. The variables were then imported into Amos to draw the theoretical model. Is this the proper procedure or should I keep the data separated and perform a factor analysis and reliability test for both samples? I have a difference in sample size (n1=50, n2=28) so there could be big differences in the loadings. How can I combine this data into one reliable model?
    After that, I can perform the multigroup analysis and chi-square test, right? Thanks for explaining how multigroup analysis works, it's much clearer now.
    The variables include measures for culture and attitude, amongst others. The research goals are to confirm a theoretical model and to compare the variables across the two samples.
    Last edited by robertz; 09-03-2010 at 09:39 AM.

  5. #5
    Phineas Packard
    Points: 16,013, Level: 81
    Level completed: 33%, Points required for next Level: 337
    Lazar's Avatar
    Location
    Sydney
    Posts
    1,159
    Thanks
    198
    Thanked 336 Times in 299 Posts
    Quote Originally Posted by robertz View Post
    raaaaaaaaaaa
    Your welcome (I think )


    Think of multi-group analysis as a test of whether culture moderates your model. ie if your model is different for different cultures.

  6. #6
    Points: 1,587, Level: 22
    Level completed: 87%, Points required for next Level: 13

    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Lol, just ignore that.
    Don't you mean nationality? Because that would be my grouping variable. Other than that I think I understand the multi-group analysis now. One thing is not very clear to me though. Each variable in the model represents the mean scores of a set of corresponding likert scales. So the variables that make up the model, whether it turns out to be moderated or not, are the mean scores of both nationalities combined or separated?

  7. #7
    Phineas Packard
    Points: 16,013, Level: 81
    Level completed: 33%, Points required for next Level: 337
    Lazar's Avatar
    Location
    Sydney
    Posts
    1,159
    Thanks
    198
    Thanked 336 Times in 299 Posts
    Quote Originally Posted by robertz View Post
    Lol, just ignore that.
    Don't you mean nationality? Because that would be my grouping variable. Other than that I think I understand the multi-group analysis now. One thing is not very clear to me though. Each variable in the model represents the mean scores of a set of corresponding likert scales. So the variables that make up the model, whether it turns out to be moderated or not, are the mean scores of both nationalities combined or separated?
    Nationality culture

    As for your second question I am not sure I follow. What means are you after?

  8. #8
    Points: 1,587, Level: 22
    Level completed: 87%, Points required for next Level: 13

    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Ok, so I measured 9 variables that are supposed to predict the dependent variable of the model, pro-environmental consumer behaviour. Variables include personal values, environmental worldview, perceived behavioural control, subjective norms, etc. I want to test whether there are differences between the two countries. If I measure mean differences with a t-test, would that be sufficient or does multi-group analysis tell me something that a t-test doesn't?

    The variables are measured by likert scales, which are computed into scores for each respondent per variable, and subsequently a mean score for the whole sample per variable. I calculated the variables, the ones that I use in Amos to draw the model with, by using data from both samples. So in Amos, the variables represent the scores of both Australians and Malaysians. Factor analysis and scale reliability tests were not consistent between the samples, so I figured that combined data was the most logical, and I managed to modify the model until I obtained an acceptable model fit.
    However, I'm afraid that I made a mistake by combining the data into one model, and that I should have kept them separated. Maybe the correct way is to draw two models in Amos and, if so, is there a way integrate them?
    I'm sorry if these questions are very basic, it's a first time thing for me

  9. #9
    Phineas Packard
    Points: 16,013, Level: 81
    Level completed: 33%, Points required for next Level: 337
    Lazar's Avatar
    Location
    Sydney
    Posts
    1,159
    Thanks
    198
    Thanked 336 Times in 299 Posts
    Quote Originally Posted by robertz View Post
    Ok, so I measured 9 variables that are supposed to predict the dependent variable of the model, pro-environmental consumer behaviour. Variables include personal values, environmental worldview, perceived behavioural control, subjective norms, etc. I want to test whether there are differences between the two countries. If I measure mean differences with a t-test, would that be sufficient or does multi-group analysis tell me something that a t-test doesn't?
    I would just do a set of t-tests if I were you. You can run something in SEM called a MIMIC model (see the book I suggested) which is like a latent variable MANOVA. But if it is not a central focus i would not bother.

    The variables are measured by likert scales, which are computed into scores for each respondent per variable, and subsequently a mean score for the whole sample per variable. I calculated the variables, the ones that I use in Amos to draw the model with, by using data from both samples. So in Amos, the variables represent the scores of both Australians and Malaysians. Factor analysis and scale reliability tests were not consistent between the samples, so I figured that combined data was the most logical, and I managed to modify the model until I obtained an acceptable model fit.
    No you are fine as I assume the scores you computed were separate for each individual. This is what the analysis is based on so it is not a problem. For the most part SEM models are based on covariance structures rather than meanstructures so for the most part you are only interested in the relationships between variables.

    I do worry that you modified your model to get an acceptable fit. Mod indices are a problematic area and I would be very very careful about using them.

    However, I'm afraid that I made a mistake by combining the data into one model, and that I should have kept them separated. Maybe the correct way is to draw two models in Amos and, if so, is there a way integrate them?
    I don't know about AMOS but no I highly doubt you would draw two models. You want to know if you single hypothesized model is the same for the two groups. This is achieved by submitting one model to the data and holding parameter estimates invariant across groups (MPLUS or lavaan in R makes the whole process of multigroup analysis so much easier).
    I'm sorry if these questions are very basic, it's a first time thing for me
    No problems. I am guessing you are from Australia. If so check out the ACSPRI courses.

    If I was in your situation i would do as follows.

    1. Descriptives, reliabilities, etc. Include a set of t-tests looking at gender differences and nationality difference on the variables of interest.

    2. Run an overarching CFA with all your variables of interest (if you have latent variables) report the correlation matrix.

    3. Test to see if that CFA is invariant across nationalities. Report invariance tests.

    4. Run your multigroup-analysis.

  10. #10
    Points: 1,587, Level: 22
    Level completed: 87%, Points required for next Level: 13

    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks! The one/two model part is clear now. All my variables are observed, so that means I can skip the overarching CFA? I'm also asking this because the scale reliabilities are all acceptable (alpha > 0.7), but the factor analysis showed loadings that didn't really make sense, such as negative loadings and overlap. The purpose of the correlation matrix is to check wether there are either negative, weak or too strong correlations, am I correct? I also checked for multicolinearity between the factors, but that was not the case.

    I'm considering the following options:
    a) follow the CFA and draw a model with the limited number of factors
    b) based on Cronbach's alpha and sound correlations between the variables, draw the model with the original factors


    Btw I studied in Australia for one semester, but I'm actually from The Netherlands. Thanks for the suggestion though

  11. #11
    Phineas Packard
    Points: 16,013, Level: 81
    Level completed: 33%, Points required for next Level: 337
    Lazar's Avatar
    Location
    Sydney
    Posts
    1,159
    Thanks
    198
    Thanked 336 Times in 299 Posts
    Quote Originally Posted by robertz View Post
    Thanks! The one/two model part is clear now. All my variables are observed, so that means I can skip the overarching CFA? I'm also asking this because the scale reliabilities are all acceptable (alpha > 0.7), but the factor analysis showed loadings that didn't really make sense, such as negative loadings and overlap.
    If you have more than one item per construct you are interested in i would strongly suggest you include this measurement structure in your model. Models with only manifest scores are biased and often underestimate the parameter estimates. You will most likely do yourself a great deal of good by going to latent variables if your predictors are not too highly correlated.

    The measurement structure in SEM is typically quite different from exploratory factor analysis as in this case you impose the measurement model on your data as such you set cross loadings to zero (typically). Again refer to Byrne's book or Schumarker and Lomax have a good introduction as well.

    The purpose of the correlation matrix is to check wether there are either negative, weak or too strong correlations, am I correct? I also checked for multicolinearity between the factors, but that was not the case.
    Well it is more to check whether the measurement structure underlying your proposed model holds and is invariant across cultures. Also the correlations give you a reference to check the parameter estimates in your model. Finally, if your model is typical poor fit typically resides in the measurement rather than the structural (your hypothesized relationships between constructs). As such it is best to explore reasons for poor fit at this level before moving to your structural model.

    I'm considering the following options:
    a) follow the CFA and draw a model with the limited number of factors
    b) based on Cronbach's alpha and sound correlations between the variables, draw the model with the original factors
    If your sample size is sufficient I would maintain the measurement structure. In other words keep the syntax from your CFA and then replace the correlation paths between latent variables with directional (or regression) paths. Then explore multigroup. If the sample size will not support this. Drop the CFA report alpha and then test your structural model. It is less than ideal but other methods such as the use of composite scores might take to long to learn if you are looking to submit soon.

    Btw I studied in Australia for one semester, but I'm actually from The Netherlands. Thanks for the suggestion though
    haha sorry your sample is just a typical Australian research sample so I guessed. If you are Dutch you are in luck! Joop Hox is a leading expert in SEM and he often runs courses. Likewise there is a very active MPLUS community over there.

  12. #12
    Points: 1,587, Level: 22
    Level completed: 87%, Points required for next Level: 13

    Posts
    9
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Another very helpful explanation, much appreciated! Ok so let's see if I understand the process correctly:

    1) I will draw each latent variable, including the observed variables, and place them next to each other in no particular order. So not in the hypothesized model structure, right?
    2) Next I connect the latent variables with double headed arrows i.e. the correlations.
    3) Calculate the estimates for both groups.

    What are the values I need to look at in the output, and how do I compare them across the two groups (the invariance test) ?

    After I correlated all my 13 variables (12 are latent) I looked at the correlations matrix for all data as one group, and didn't find anything that looked very unusual. A couple of correlations were not in line with the theory, so is there a way to adjust them? And do I need to do that first before I move on to testing for invariance? The output noted a covariance matrix with all 13 factors as not being positive definite.


    UPDATE => I managed to fix the correlations, so the correlation matrix looks fine. However the non-positive definite coviarance matrix was still reported. Is this something to worry about (and to fix) or should I just focus on the correlation matrix for now?
    Last edited by robertz; 09-08-2010 at 06:21 AM.

  13. #13
    Phineas Packard
    Points: 16,013, Level: 81
    Level completed: 33%, Points required for next Level: 337
    Lazar's Avatar
    Location
    Sydney
    Posts
    1,159
    Thanks
    198
    Thanked 336 Times in 299 Posts

    Re: Multiple Group Analysis necessary?

    Quote Originally Posted by robertz View Post
    Another very helpful explanation, much appreciated! Ok so let's see if I understand the process correctly:

    1) I will draw each latent variable, including the observed variables, and place them next to each other in no particular order. So not in the hypothesized model structure, right?
    2) Next I connect the latent variables with double headed arrows i.e. the correlations.
    3) Calculate the estimates for both groups.

    What are the values I need to look at in the output, and how do I compare them across the two groups (the invariance test) ?

    After I correlated all my 13 variables (12 are latent) I looked at the correlations matrix for all data as one group, and didn't find anything that looked very unusual. A couple of correlations were not in line with the theory, so is there a way to adjust them? And do I need to do that first before I move on to testing for invariance? The output noted a covariance matrix with all 13 factors as not being positive definite.


    UPDATE => I managed to fix the correlations, so the correlation matrix looks fine. However the non-positive definite coviarance matrix was still reported. Is this something to worry about (and to fix) or should I just focus on the correlation matrix for now?
    Not sure how to specify models in AMOS and without looking at the output I can not really guide you much here. Yes a non-positive definite matrix is a problem but without looking at it I would not be able to tell you what has gone wrong. Maybe it is time to consult someone local that can sit with you as you specify your models.

    If you have specified the model correctly you may have to provide better starting values. Again I cannot tell you what they might be without looking at it and given I don't know AMOS I probably can not tell you how to specify those starting values anyway.

    Sorry.

    EDIT: If you have a way of posting the output here I might be able to look at whether you have specified your model correctly.
    Last edited by Lazar; 09-09-2010 at 01:48 AM.

  14. #14
    Points: 227, Level: 4
    Level completed: 54%, Points required for next Level: 23

    Location
    USA
    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Multiple Group Analysis


    I am going to perform MCFA. Is there a sample size requirement for such analysis or are there any references regarding sample size for MCFA?

    Any comments are appreciated.

    Thanks

+ Reply to Thread

           




Similar Threads

  1. Replies: 4
    Last Post: 08-06-2010, 06:15 AM
  2. 3-group analysis
    By yooguruto in forum Biostatistics
    Replies: 0
    Last Post: 07-26-2010, 01:25 PM
  3. One-group pretest-posttest analysis?
    By FoosJunkie in forum Statistics
    Replies: 1
    Last Post: 04-02-2010, 06:28 PM
  4. Help needed: multiple test in a small size group
    By sandsoppa in forum Other Software
    Replies: 1
    Last Post: 09-25-2009, 09:50 PM
  5. Group Analysis- Help with interpretation
    By anna_s in forum Statistics
    Replies: 0
    Last Post: 01-29-2009, 05:47 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats