Thread: Does covariate must be observed variable?

1. Does covariate must be observed variable?

In my study, I found significant interactions between treatment and order of sessions. Can I put the order variable as covariate in order to control this effect?

2. Re: Does covariate must be observed variable?

Session?

Describe a little more but likely, yes. Did you record session or between session length.

3. Re: Does covariate must be observed variable?

Hi! I randomly assigned participants into two orders: The first group received the intervention in the first session and the placebo in the second session, while the second group received the intervention in the second session and the placebo in the first session. I found significant interaction between treatment and order: the first group scored higher in the intervention, and the second group scored higher in the placebo.
My question is: Can I "control" the effects of order by entering the order variable as a covariate?

4. Re: Does covariate must be observed variable?

Absolutely. Seems like a crossover design. Glad to hear you randomized

5. Re: Does covariate must be observed variable?

Originally Posted by PsychologyS
Hi! I randomly assigned participants into two orders: The first group received the intervention in the first session and the placebo in the second session, while the second group received the intervention in the second session and the placebo in the second session. I found significant interaction between treatment and order: the first group scored higher in the intervention, and the second group scored higher in the placebo.
hi,
doesn't this look like everybody performed better the first time? I don't really see an interaction here.

regards

6. Re: Does covariate must be observed variable?

hlsmith- Thank you for your response!

rogojel- that is correct. the interaction is between treatment (intervention, placebo) to order (intervention first (1), placebo first (2))- for scores for order 1, was higher in the intervention, and the scores for order 2, was higher in the placebo. the meaning, as you say, is that order had effect on the performance. I want to "clean" the effect of order to see the effect of the intervention.

7. Re: Does covariate must be observed variable?

How about a simpler model where only the order has an effect?

8. Re: Does covariate must be observed variable?

Originally Posted by rogojel
How about a simpler model where only the order has an effect?
I'm not interested in the order effect- I assumed there will be order effect, that's the reason I counterbalanced and tried to avoid them.
I'm interested in the effect of intervention. When I put the order as a covariate, I find significant effect of the treatment.

9. Re: Does covariate must be observed variable?

Originally Posted by PsychologyS
I'm not interested in the order effect- I assumed there will be order effect, that's the reason I counterbalanced and tried to avoid them.
I'm interested in the effect of intervention. When I put the order as a covariate, I find significant effect of the treatment.
As you noted before, you should include order as a covariate if you believe it has an actual impact on the dependent variable. You don't need to take much direct interest in it, necessarily. I probably would pay little attention to it if you have a strong theoretical reason to include the order. That is, I wouldn't bother significance testing the order covariate if there is no genuine interest in the order variable aside from trying to estimate other effects. You can avoid any testing errors and the theory/logic should support the order variable's inclusion in the model to properly estimate the other effects. Of course, if there is some interest in the order variable, you can do some testing or interval estimation to learn a bit more about it.

The simplest example I can think of is this: predicting weight in infants as a function of their age in months. A reasonable covariate to include would be the infant's length measurement (similar to accounting for height of someone a bit older). You may not be interested in the effect of length on weight, but to estimate the effect of age on infant weight, length should be accounted for. It is possible an interaction is needed between length and age, but this should be left to your good judgement, theory, and prior literature in most cases.

10. Re: Does covariate must be observed variable?

Originally Posted by ondansetron
As you noted before, you should include order as a covariate if you believe it has an actual impact on the dependent variable. You don't need to take much direct interest in it, necessarily. I probably would pay little attention to it if you have a strong theoretical reason to include the order. That is, I wouldn't bother significance testing the order covariate if there is no genuine interest in the order variable aside from trying to estimate other effects. You can avoid any testing errors and the theory/logic should support the order variable's inclusion in the model to properly estimate the other effects. Of course, if there is some interest in the order variable, you can do some testing or interval estimation to learn a bit more about it.

The simplest example I can think of is this: predicting weight in infants as a function of their age in months. A reasonable covariate to include would be the infant's length measurement (similar to accounting for height of someone a bit older). You may not be interested in the effect of length on weight, but to estimate the effect of age on infant weight, length should be accounted for. It is possible an interaction is needed between length and age, but this should be left to your good judgement, theory, and prior literature in most cases.
Thanks for your reply! I agree with you. Becasue I randomly assigned the order, the only thing I worried about is the defintion of covariate in some places as observed variable (not manipulated variable).

Also- When I enter the order variable as covariate or as factor, it changes the sum of squares and my F value. Why there are difference? By using covariate, I measure the difference between the levels of my independent variable in the average levels of order. It is not exactly what happens when using the order effect as factor and looking for main effect?

11. Re: Does covariate must be observed variable?

Originally Posted by PsychologyS
Thanks for your reply! I agree with you. Becasue I randomly assigned the order, the only thing I worried about is the defintion of covariate in some places as observed variable (not manipulated variable).

Also- When I enter the order variable as covariate or as factor, it changes the sum of squares and my F value. Why there is difference? By using covariate, I measure the difference between the levels of my independent variable in the average levels of order. It is not exactly what happens when using the order effect as factor and looking for main effect?
Sure, I can see your point. I was just using the term a bit loosely in the sense that you think order impacts the DV but you aren't primarily interested in the order variable. However, due to the effect you believe it has, you're going to include it just to make sure your estimation of other parameters is as optimal as possible for your study. If you know that no matter who is in group 1, they will perform better than group 2, for example, even by randomization, I think you should include the order covariate, although there may be a better approach to this that I haven't yet thought of.

For your second question, I believe the only real difference is how you, the researcher, will choose to interpret the output. The mathematics is the same in this case, but you are aware you're treating the order as a covariate rather than a variable of interest. I think some software packages separate out "independent variables/factors" from "covariates" to help researchers keep things separate in their mind.

I fairly certain of this, but maybe someone else can clarify if I misunderstood something.

12. Re: Does covariate must be observed variable?

Originally Posted by ondansetron
Sure, I can see your point. I was just using the term a bit loosely in the sense that you think order impacts the DV but you aren't primarily interested in the order variable. However, due to the effect you believe it has, you're going to include it just to make sure your estimation of other parameters is as optimal as possible for your study.

For your second question, I believe the only real difference is how you, the researcher, will choose to interpret the output. The mathematics is the same in this case, but you are aware you're treating the order as a covariate rather than a variable of interest. I think some software packages separate out "independent variables/factors" from "covariates" to help researchers keep things separate in their mind.

I fairly certain of this, but maybe someone else can clarify if I misunderstood something.
I understand that the main difference is in the intereption of the output, but wonder why the results are different if I put the variable as covariate or as factor, if the mathmatics is the same (the intervention is significant if I put the variable as covariate, but only marginal significant if I put the variable as factor).

Maybe it relates to the manner I entered the order variable? It's a catagorial variable 0f 0,1 - I just entered it as it is.

13. Re: Does covariate must be observed variable?

Originally Posted by ondansetron
Sure, I can see your point. I was just using the term a bit loosely in the sense that you think order impacts the DV but you aren't primarily interested in the order variable. However, due to the effect you believe it has, you're going to include it just to make sure your estimation of other parameters is as optimal as possible for your study. If you know that no matter who is in group 1, they will perform better than group 2, for example, even by randomization, I think you should include the order covariate, although there may be a better approach to this that I haven't yet thought of.

For your second question, I believe the only real difference is how you, the researcher, will choose to interpret the output. The mathematics is the same in this case, but you are aware you're treating the order as a covariate rather than a variable of interest. I think some software packages separate out "independent variables/factors" from "covariates" to help researchers keep things separate in their mind.

I fairly certain of this, but maybe someone else can clarify if I misunderstood something.
By the way, I didn't find main effect for order, but I find interaction between order and treatment. Entering the order as covariate still deal with it?

the graph of the interaction is attached- the horizantal axis represent the intervention, and the lines represent the order.

14. Re: Does covariate must be observed variable?

The scale of the y axis will make a visual difference! Can you slap confidence whiskers on the plots and post the model output to clear things up.

15. Re: Does covariate must be observed variable?

Originally Posted by PsychologyS
Hi! I randomly assigned participants into two orders: The first group received the intervention in the first session and the placebo in the second session, while the second group received the intervention in the second session and the placebo in the first session. I found significant interaction between treatment and order: the first group scored higher in the intervention, and the second group scored higher in the placebo.
My question is: Can I "control" the effects of order by entering the order variable as a covariate?
Hi all, just a quick response and I haven't carefully read all the replies, but... If order is randomly assigned, you do not need to control for it to accurately estimate the effect of treatment. (You don't need to control for everything that affects the DV, only things that are correlated with the IV and that affect the DV - i.e. confounding variables). It would frequently be the case that order affects the DV, but the entire point of counterbalancing with random assignment to orders is so that we don't have to worry about order effects confounding the results.

Note also that if you include main effects or treatment and order plus their interaction, the main effect of treatment now means something different - i.e., it is now the effect of treatment for people in just one of the orders, not both. This is no longer estimating the effect you are interested in.

If the effect of order was of genuine substantive interest in of itself you could include it, but that's clearly not the case here.

When I put the order as a covariate, I find significant effect of the treatment.
Choosing a model specification on the basis that it produces the result you want to see is actually p-hacking and a really bad idea (though I'm sure that wasn't your intention). I strongly suggest that you report the model you originally planned, not this one. In future pre-register your analysis plans before collecting data; that way readers of your research can have faith that the results represent a fair test of the hypothesis of interest.