+ Reply to Thread
Results 1 to 14 of 14

Thread: Let me know a better analysis method?

  1. #1
    Points: 54, Level: 1
    Level completed: 8%, Points required for next Level: 46

    Posts
    8
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Question Let me know a better analysis method?




    I have two IVs; "collection method" (online and classroom) and "culture" (A and B), one continuous variable ("Age") and 8 DVs (well-being variables and self-construal variables). I wanted to see if the two cultures were different in those DVs controlling for "collection method" and "Age".
    My approach was to conduct 2 way MANCOVA with the two IVs, covariate on the 8 DVs. However, a reviewer questioned the problem of multiple t-tests and wondered about the simpler and more valid method.
    Could anyone help me on this issue?

  2. #2
    TS Contributor
    Points: 17,749, Level: 84
    Level completed: 80%, Points required for next Level: 101
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,540
    Thanks
    56
    Thanked 639 Times in 601 Posts

    Re: Let me know a better analysis method?

    Could you perhaps describe the reviewer's comment in more detail?
    Since you want to perform a MANOVA, even including a covariate,
    there are no t-tests. Therefore, the critique is a bit difficult to understand.

    With kind regards

    Karabiner
    »Jetzt kann mich der Führer mal am Arsch lecken.« (Ernst Kuzorra, 1941)

  3. #3
    Points: 3, Level: 1
    Level completed: 5%, Points required for next Level: 47

    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Let me know a better analysis method?

    Action Required to Activate Membership for Statistics Help @ Talk Stats Forum
    http://rongbachkim888.xtgem.com

  4. #4
    Points: 54, Level: 1
    Level completed: 8%, Points required for next Level: 46

    Posts
    8
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: Let me know a better analysis method?

    Thank you Karabiner. Sorry that I failed to post the detailed description. After the MAN(C)OVA analysis, I found that not "culture" but "collection method" was effective inconsistent with my hypothesis. Also age was effective and so for the interaction between culture and collection method on a few DVs. But in order to test the hypothesis, I conducted separate analyses for the two collection method groups (classroom and online) with multiple t-tests and found that the hypothesis was supported only in classroom sample. As the two groups were quite different in terms of average age (classroom younger and online older), I additionally conducted correlational analyses and found that the inconsistencies may be related with age. ...
    Is my approach wrong? The reviewer says it just "feels" that this was not good and asked me to get advice from an expert. If you want, I'm happy to send the manuscript and comments. Thanks!!

  5. #5
    TS Contributor
    Points: 17,749, Level: 84
    Level completed: 80%, Points required for next Level: 101
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,540
    Thanks
    56
    Thanked 639 Times in 601 Posts

    Re: Let me know a better analysis method?

    Also age was effective and so for the interaction between culture and collection method on a few DVs.
    Where do these DVs suddenly come from? You have 1 dependent variable, a linear construct
    composed from 8 variables. What else was analysed as dependent variables?

    But in order to test the hypothesis
    The description is a bit unclear. I am not sure what you hypothesis is.
    I suppose the hypothesis was not introducedvafter you had analysed the data?

    With kind regards

    Karabiner
    »Jetzt kann mich der Führer mal am Arsch lecken.« (Ernst Kuzorra, 1941)

  6. #6
    Points: 54, Level: 1
    Level completed: 8%, Points required for next Level: 46

    Posts
    8
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: Let me know a better analysis method?

    Hi Karabiner,

    As I wrote in the first posting, there were 8 DVs I wanted to check "country" (my interest) and "collection method" (not my interest) effects on each. But the MANCOVA test found "country" was not significant overall whereas "collection method" was significant for some of the variables. There were also significant interaction effects between "age" and "collection method" on some variables. "Age" was not independent from "collection method" in the study design (average age was higher in online sample than classroom sample and age variance was greater in online sample too). Anyway so in order to test my hypothesis that the mean value of a target DV would be different between countries, I conducted separate sets of independent sample t-tests by collection method (between classroom samples and between online samples). The hypothesis was supported in classroom samples only.
    And the hypothesis is introduced beforehand in Introduction.
    Can you understand what I did? Don't you mind if I sent the materials to you in case? I think that'd be easier to understand. Thanks!!

  7. #7
    TS Contributor
    Points: 17,749, Level: 84
    Level completed: 80%, Points required for next Level: 101
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,540
    Thanks
    56
    Thanked 639 Times in 601 Posts

    Re: Let me know a better analysis method?

    As I wrote in the first posting, there were 8 DVs I wanted to check "country" (my interest) and "collection method" (not my interest) effects on each. But the MANCOVA test found "country" was not significant overall whereas "collection method" was significant for some of the variables.
    This is confusing. If you performed a multivariate analysis of variance (with an additional covariate) = MANCOVA, then you had 1 dependent variable which was composed from 8 individual variables. If, instead, you wanted to analyse the 8 DVs independently from each other, then you would not have performed a MAN(C)OVA.

    With kind regards

    Karabiner
    »Jetzt kann mich der Führer mal am Arsch lecken.« (Ernst Kuzorra, 1941)

  8. #8
    Points: 54, Level: 1
    Level completed: 8%, Points required for next Level: 46

    Posts
    8
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: Let me know a better analysis method?

    Then what would be the appropriate method? Sorry for the confusion.

  9. #9
    Points: 54, Level: 1
    Level completed: 8%, Points required for next Level: 46

    Posts
    8
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: Let me know a better analysis method?

    I thought manova is used when there are multiple DVs.

  10. #10
    TS Contributor
    Points: 17,749, Level: 84
    Level completed: 80%, Points required for next Level: 101
    Karabiner's Avatar
    Location
    FC Schalke 04, Germany
    Posts
    2,540
    Thanks
    56
    Thanked 639 Times in 601 Posts

    Re: Let me know a better analysis method?

    As far as I know and from what I have learned, MANOVA is used if you have a set of dependent variables which jointly represent a hypothetical construct. Therefore, one decides whether this is the case (1 construct, MANOVA, no analyses of single DVs), or whether one is interested in the single DVs (no need for a MANOVA then). If you are interested in analysing 9 separate variables and want to perform group comparisons, then honestly I do not know a clever method to avoid multiple testing. You could adjust the level of signficance, of course, if type 1 errors were an issue here.

    With kind regards

    Karabiner
    »Jetzt kann mich der Führer mal am Arsch lecken.« (Ernst Kuzorra, 1941)

  11. The Following User Says Thank You to Karabiner For This Useful Post:

    joonha35 (06-19-2017)

  12. #11
    Points: 54, Level: 1
    Level completed: 8%, Points required for next Level: 46

    Posts
    8
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: Let me know a better analysis method?

    Hi Karabiner,
    thanks for the comment. 4 out of the 8 variables can be said to represent a hypothetical construct (i.e., individual self, collective self and two types of relational self) whereas the other four variables are different constructs. In this case, may I conduct MANCOVA with only 4 DVs as a single construct?

  13. #12
    Points: 1,741, Level: 24
    Level completed: 41%, Points required for next Level: 59

    Posts
    230
    Thanks
    37
    Thanked 68 Times in 59 Posts

    Re: Let me know a better analysis method?

    Quote Originally Posted by Karabiner View Post
    As far as I know and from what I have learned, MANOVA is used if you have a set of dependent variables which jointly represent a hypothetical construct.
    You are right that this is one application for MANOVA, but more generally, it can be employed when the DVs have an underlying covariance structure. To me your answer sounds more pointed towards use only in psychological/sociological studies; perhaps I have misinterpreted your meaning.

    However, take for example, the DVs of body weight, mean arterial pressure, and LDL cholesterol measured on each patient as a function of some treatment. There is a clear rationale to support a relationship between these three DVs, so a MANOVA may be an appropriate test early on. However, if there is sufficient evidence of a difference in the mean vectors across treatment groups, you can use post-hoc type analyses to see where the differences exist between treatment groups-- ANCOVAs are often applicable, after the significant MANOVA, in this case to still account for the underlying covariance structure of the 3 DVs. If you are investigating body weight in one ANCOVA, you'd also include the mean arterial pressure and LDL cholesterol variables on the RHS as covariates, while still looking at the treatment as your independent variable of interest.

    This all is highly dependent on your research questions, of course.
    Hope this helps.

  14. The Following 2 Users Say Thank You to ondansetron For This Useful Post:

    joonha35 (06-21-2017), Karabiner (06-19-2017)

  15. #13
    Points: 54, Level: 1
    Level completed: 8%, Points required for next Level: 46

    Posts
    8
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: Let me know a better analysis method?

    Thank you Karabiner,
    it is very helpful. Maybe I could consider the four different self variables (variables A, B, C, D) as a single construct? To rephrase my hypothesis, it was that the degrees of variable C would be different between two countries.
    Following your suggestion, I first inserted only "collection method" and "country" as IVs and the four self variables as DVs (a single construct) without "Age" covariate. It was found that "collection method" and "collection method x country" but not "country" were significant (I mainly read Wilk's Lambda throughout the result tables although it is not so different from other tests). Then I conducted MANCOVA including "Age" as a covariate and this time found that only "collection method x country" was significant. "Age" could have done some role here but the single effect itself was not significant.
    Did I do right so far?
    Last edited by joonha35; 06-21-2017 at 09:08 PM.

  16. #14
    Points: 54, Level: 1
    Level completed: 8%, Points required for next Level: 46

    Posts
    8
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: Let me know a better analysis method?


    Finally, after considering alternative methods, I've ended up with conducting 3-way repeated measures ANCOVA as described below to test the hypothesis that there would be a country group difference in one of the self variables when age was controlled (i did not expect collection method effect).
    Could you please have a look if this approach would be valid? And in this case what should I do further? separate ANCOVA in each collection method?
    A 3-way (self, country, and collection method) repeated measures ANCOVA was conducted with age as a covariance. Mauchly’s test x2(5) =136.62, p < .001 indicated that sphericity was not assumed. The following results are based on Huynh-Feldt, as Greenhouse-Geisser Epsilon was .81, which was above the criterion value (.75, Field, 2013; Howell, 2002). There was a self main effect, F(2.5, 1221)=12.59, p < .001 and interaction effect between self and age, F(2.5, 1221)= 12.83, p < .001, indicating that dominant self-construals varied across age. Interaction between self and country was not significant, F(2.5, 1221) = .1.5, p = .22, indicating that there was no significant difference between Japanese and Korean groups in ratings on different selves. However, there was a marginal interaction effect between self and collection method, F(2.5, 1221) = .2.17, p = .10 and also a three-way interaction between self, country, and collection method, F(2.5, 1221) = .2.63, p = .06.

    Field, A. (2013). Discovering Statistics with IBM SPSS Newbury Park, CA: Sage.
    Howell, D.C. (2002). Statistical Methods for Psychology (5th ed.). Pacific Grove CA: Duxbury.

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats