+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 15 of 22

Thread: Does covariate must be observed variable?

Hybrid View

  1. #1
    Points: 311, Level: 6
    Level completed: 22%, Points required for next Level: 39

    Posts
    14
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Does covariate must be observed variable?

    In my study, I found significant interactions between treatment and order of sessions. Can I put the order variable as covariate in order to control this effect?

  2. #2
    Omega Contributor
    Points: 38,284, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,991
    Thanks
    397
    Thanked 1,185 Times in 1,146 Posts

    Re: Does covariate must be observed variable?

    Session?

    Describe a little more but likely, yes. Did you record session or between session length.
    Stop cowardice, ban guns!

  3. #3
    Points: 311, Level: 6
    Level completed: 22%, Points required for next Level: 39

    Posts
    14
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Does covariate must be observed variable?

    Hi! I randomly assigned participants into two orders: The first group received the intervention in the first session and the placebo in the second session, while the second group received the intervention in the second session and the placebo in the first session. I found significant interaction between treatment and order: the first group scored higher in the intervention, and the second group scored higher in the placebo.
    My question is: Can I "control" the effects of order by entering the order variable as a covariate?
    Last edited by PsychologyS; 07-04-2017 at 01:11 AM.

  4. #4
    TS Contributor
    Points: 12,227, Level: 72
    Level completed: 45%, Points required for next Level: 223
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,470
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Does covariate must be observed variable?

    Quote Originally Posted by PsychologyS View Post
    Hi! I randomly assigned participants into two orders: The first group received the intervention in the first session and the placebo in the second session, while the second group received the intervention in the second session and the placebo in the second session. I found significant interaction between treatment and order: the first group scored higher in the intervention, and the second group scored higher in the placebo.
    hi,
    doesn't this look like everybody performed better the first time? I don't really see an interaction here.

    regards

  5. #5
    Points: 311, Level: 6
    Level completed: 22%, Points required for next Level: 39

    Posts
    14
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Does covariate must be observed variable?

    hlsmith- Thank you for your response!

    rogojel- that is correct. the interaction is between treatment (intervention, placebo) to order (intervention first (1), placebo first (2))- for scores for order 1, was higher in the intervention, and the scores for order 2, was higher in the placebo. the meaning, as you say, is that order had effect on the performance. I want to "clean" the effect of order to see the effect of the intervention.
    Last edited by PsychologyS; 07-04-2017 at 01:18 AM.

  6. #6
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: Does covariate must be observed variable?

    Quote Originally Posted by PsychologyS View Post
    Hi! I randomly assigned participants into two orders: The first group received the intervention in the first session and the placebo in the second session, while the second group received the intervention in the second session and the placebo in the first session. I found significant interaction between treatment and order: the first group scored higher in the intervention, and the second group scored higher in the placebo.
    My question is: Can I "control" the effects of order by entering the order variable as a covariate?
    Hi all, just a quick response and I haven't carefully read all the replies, but... If order is randomly assigned, you do not need to control for it to accurately estimate the effect of treatment. (You don't need to control for everything that affects the DV, only things that are correlated with the IV and that affect the DV - i.e. confounding variables). It would frequently be the case that order affects the DV, but the entire point of counterbalancing with random assignment to orders is so that we don't have to worry about order effects confounding the results.

    Note also that if you include main effects or treatment and order plus their interaction, the main effect of treatment now means something different - i.e., it is now the effect of treatment for people in just one of the orders, not both. This is no longer estimating the effect you are interested in.

    If the effect of order was of genuine substantive interest in of itself you could include it, but that's clearly not the case here.

    When I put the order as a covariate, I find significant effect of the treatment.
    Choosing a model specification on the basis that it produces the result you want to see is actually p-hacking and a really bad idea (though I'm sure that wasn't your intention). I strongly suggest that you report the model you originally planned, not this one. In future pre-register your analysis plans before collecting data; that way readers of your research can have faith that the results represent a fair test of the hypothesis of interest.
    Matt aka CB | twitter.com/matthewmatix

  7. #7
    Points: 311, Level: 6
    Level completed: 22%, Points required for next Level: 39

    Posts
    14
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Does covariate must be observed variable?

    Quote Originally Posted by CowboyBear View Post
    Hi all, just a quick response and I haven't carefully read all the replies, but... If order is randomly assigned, you do not need to control for it to accurately estimate the effect of treatment. (You don't need to control for everything that affects the DV, only things that are correlated with the IV and that affect the DV - i.e. confounding variables). It would frequently be the case that order affects the DV, but the entire point of counterbalancing with random assignment to orders is so that we don't have to worry about order effects confounding the results.

    Note also that if you include main effects or treatment and order plus their interaction, the main effect of treatment now means something different - i.e., it is now the effect of treatment for people in just one of the orders, not both. This is no longer estimating the effect you are interested in.

    If the effect of order was of genuine substantive interest in of itself you could include it, but that's clearly not the case here.



    Choosing a model specification on the basis that it produces the result you want to see is actually p-hacking and a really bad idea (though I'm sure that wasn't your intention). I strongly suggest that you report the model you originally planned, not this one. In future pre-register your analysis plans before collecting data; that way readers of your research can have faith that the results represent a fair test of the hypothesis of interest.
    Thanks for your reply. The counterbalance deal with the order effect (there was no main effect for order) but still there is a interaction between order and treatment in way that mask the effect of treatment.
    You wrote that I need to control "only things that are correlated with the IV and that affect the DV"- as far as I know, covariate cannot be variable that correlate with the IV (this is one of the assumption in ANCOVA)

    Alsom you wrote that main effect of treatment now means the effect of treatment for people in just one of the orders, not both- I don't understand this- the effect of treatments means the effect of treatment in the average level of order (I checked it in the descriptive).

    Before I analyzed the data, I assume there will be interaction between order and session, and I wanted to "clean" the effect of order by putting it in the between-subject factor or in the covariate factor. Because i'm not interested in the order itself, I decided to put it in the covariate. What do you think?

  8. #8
    TS Contributor
    Points: 18,889, Level: 87
    Level completed: 8%, Points required for next Level: 461
    CowboyBear's Avatar
    Location
    New Zealand
    Posts
    2,062
    Thanks
    121
    Thanked 427 Times in 328 Posts

    Re: Does covariate must be observed variable?

    Quote Originally Posted by PsychologyS View Post
    Thanks for your reply. The counterbalance deal with the order effect (there was no main effect for order) but still there is a interaction between order and treatment in way that mask the effect of treatment.
    If the treatment actually has a main effect, and your design has adequate power, you would still be able to detect this effect without specifying the order*treatment interaction.


    You wrote that I need to control "only things that are correlated with the IV and that affect the DV"- as far as I know, covariate cannot be variable that correlate with the IV (this is one of the assumption in ANCOVA)
    This is not an assumption of ANCOVA. The assumptions of regression models (including ANCOVA) are covered in this paper: http://pareonline.net/getvn.asp?v=18&n=11

    Btw, I'm a little confused as to how you're including an interaction if you're specifying order as a covariate in SPSS(?) I would've thought you needed to include order in the fixed effect field if you wanted to specify the interaction. Are you sure your model includes the interaction term?

    Alsom you wrote that main effect of treatment now means the effect of treatment for people in just one of the orders, not both- I don't understand this- the effect of treatments means the effect of treatment in the average level of order (I checked it in the descriptive).
    For a default specification, the main effect of treatment (in a model including order and order*treatment) would be the effect of treatment for participants in the first level of order. It is possible to set up your coding such that the main effect is displayed for a participant with an "average level of order", but SPSS wouldn't be doing this by default as far as I know. Even if coded that way, it wouldn't be the same thing as the average effect of the treatment (which is presumably what you are most interested in).

    Before I analyzed the data, I assume there will be interaction between order and session, and I wanted to "clean" the effect of order by putting it in the between-subject factor or in the covariate factor. Because i'm not interested in the order itself, I decided to put it in the covariate. What do you think?
    You don't need to "clean" the effect of order - your use of randomisation is enough for an unbiased estimate of the effect of treatment. I would suggest that you just estimate the main effect and exclude order (though you can show descriptive statistics displaying the unusual pattern of change).
    Matt aka CB | twitter.com/matthewmatix

  9. #9
    Points: 1,741, Level: 24
    Level completed: 41%, Points required for next Level: 59

    Posts
    230
    Thanks
    37
    Thanked 68 Times in 59 Posts

    Re: Does covariate must be observed variable?

    Quote Originally Posted by CowboyBear;201749If order is randomly assigned, [B
    you do not need to control for it to accurately estimate the effect of treatment[/B]. (You don't need to control for everything that affects the DV, only things that are correlated with the IV and that affect the DV - i.e. confounding variables). It would frequently be the case that order affects the DV, but the entire point of counterbalancing with random assignment to orders is so that we don't have to worry about order effects confounding the results.
    I definitely agree with your post. I also may not have understood what OP meant nor articulated clearly what I meant. I took OP to mean that randomization will not reduce a systematic issue as it normally does.

  10. #10
    Omega Contributor
    Points: 38,284, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,991
    Thanks
    397
    Thanked 1,185 Times in 1,146 Posts

    Re: Does covariate must be observed variable?

    Absolutely. Seems like a crossover design. Glad to hear you randomized
    Stop cowardice, ban guns!

  11. #11
    TS Contributor
    Points: 12,227, Level: 72
    Level completed: 45%, Points required for next Level: 223
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,470
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Does covariate must be observed variable?

    How about a simpler model where only the order has an effect?

  12. #12
    Points: 311, Level: 6
    Level completed: 22%, Points required for next Level: 39

    Posts
    14
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Does covariate must be observed variable?

    Quote Originally Posted by rogojel View Post
    How about a simpler model where only the order has an effect?
    I'm not interested in the order effect- I assumed there will be order effect, that's the reason I counterbalanced and tried to avoid them.
    I'm interested in the effect of intervention. When I put the order as a covariate, I find significant effect of the treatment.

  13. #13
    Points: 1,741, Level: 24
    Level completed: 41%, Points required for next Level: 59

    Posts
    230
    Thanks
    37
    Thanked 68 Times in 59 Posts

    Re: Does covariate must be observed variable?

    Quote Originally Posted by PsychologyS View Post
    I'm not interested in the order effect- I assumed there will be order effect, that's the reason I counterbalanced and tried to avoid them.
    I'm interested in the effect of intervention. When I put the order as a covariate, I find significant effect of the treatment.
    As you noted before, you should include order as a covariate if you believe it has an actual impact on the dependent variable. You don't need to take much direct interest in it, necessarily. I probably would pay little attention to it if you have a strong theoretical reason to include the order. That is, I wouldn't bother significance testing the order covariate if there is no genuine interest in the order variable aside from trying to estimate other effects. You can avoid any testing errors and the theory/logic should support the order variable's inclusion in the model to properly estimate the other effects. Of course, if there is some interest in the order variable, you can do some testing or interval estimation to learn a bit more about it.

    The simplest example I can think of is this: predicting weight in infants as a function of their age in months. A reasonable covariate to include would be the infant's length measurement (similar to accounting for height of someone a bit older). You may not be interested in the effect of length on weight, but to estimate the effect of age on infant weight, length should be accounted for. It is possible an interaction is needed between length and age, but this should be left to your good judgement, theory, and prior literature in most cases.

  14. #14
    Points: 311, Level: 6
    Level completed: 22%, Points required for next Level: 39

    Posts
    14
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Does covariate must be observed variable?

    Quote Originally Posted by ondansetron View Post
    As you noted before, you should include order as a covariate if you believe it has an actual impact on the dependent variable. You don't need to take much direct interest in it, necessarily. I probably would pay little attention to it if you have a strong theoretical reason to include the order. That is, I wouldn't bother significance testing the order covariate if there is no genuine interest in the order variable aside from trying to estimate other effects. You can avoid any testing errors and the theory/logic should support the order variable's inclusion in the model to properly estimate the other effects. Of course, if there is some interest in the order variable, you can do some testing or interval estimation to learn a bit more about it.

    The simplest example I can think of is this: predicting weight in infants as a function of their age in months. A reasonable covariate to include would be the infant's length measurement (similar to accounting for height of someone a bit older). You may not be interested in the effect of length on weight, but to estimate the effect of age on infant weight, length should be accounted for. It is possible an interaction is needed between length and age, but this should be left to your good judgement, theory, and prior literature in most cases.
    Thanks for your reply! I agree with you. Becasue I randomly assigned the order, the only thing I worried about is the defintion of covariate in some places as observed variable (not manipulated variable).

    Also- When I enter the order variable as covariate or as factor, it changes the sum of squares and my F value. Why there are difference? By using covariate, I measure the difference between the levels of my independent variable in the average levels of order. It is not exactly what happens when using the order effect as factor and looking for main effect?
    Last edited by PsychologyS; 07-04-2017 at 07:16 AM.

  15. #15
    Points: 1,741, Level: 24
    Level completed: 41%, Points required for next Level: 59

    Posts
    230
    Thanks
    37
    Thanked 68 Times in 59 Posts

    Re: Does covariate must be observed variable?

    Quote Originally Posted by PsychologyS View Post
    Thanks for your reply! I agree with you. Becasue I randomly assigned the order, the only thing I worried about is the defintion of covariate in some places as observed variable (not manipulated variable).

    Also- When I enter the order variable as covariate or as factor, it changes the sum of squares and my F value. Why there is difference? By using covariate, I measure the difference between the levels of my independent variable in the average levels of order. It is not exactly what happens when using the order effect as factor and looking for main effect?
    Sure, I can see your point. I was just using the term a bit loosely in the sense that you think order impacts the DV but you aren't primarily interested in the order variable. However, due to the effect you believe it has, you're going to include it just to make sure your estimation of other parameters is as optimal as possible for your study. If you know that no matter who is in group 1, they will perform better than group 2, for example, even by randomization, I think you should include the order covariate, although there may be a better approach to this that I haven't yet thought of.

    For your second question, I believe the only real difference is how you, the researcher, will choose to interpret the output. The mathematics is the same in this case, but you are aware you're treating the order as a covariate rather than a variable of interest. I think some software packages separate out "independent variables/factors" from "covariates" to help researchers keep things separate in their mind.

    I fairly certain of this, but maybe someone else can clarify if I misunderstood something.
    Last edited by ondansetron; 07-04-2017 at 07:25 AM.

+ Reply to Thread
Page 1 of 2 1 2 LastLast

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats