Average marginal effects in logit model

[LarsG]

Hi
I have performed two logistic regression models (full model and stepwise reduced model). I have 8 predictors and 5 predictors, respectively.
Both continouos and categorical predictors and binary outcome.

Now I want to calculate average marginal effects, but will it only make sense to do it on these two models or can I also choose two variables that are clinically related and see what their AME is in a much simpler model of 2 predictors? Will it make sense to just pick relevant predictors and calculate AME two and two for example?

Do I risk to make some wrong assumptions by excluding predictors that I know are explaining some of the outcome?
My reason for doing it is because it is very hard to make clinical sense of 8 predictors in combination.

thanks

 

[hlsmith]

You need to write out a protocol before ever looking at data based on your content knowledge and stick to it regardless of the results. Otherwise you risk false discovery. What was your original plan? This is is even more important in health care since practices could be influenced by results.

PS, if it is difficult to make sense of predictors, why were they in the model to start with?

 

[LarsG]

My protocol was the logistic regression I described and that will be reported of course.
What I was considering was to describe the interrelationship between two of the predictors. This is exploratory only.

ps. It is not difficult to make sense of the predictors in respect to outcome. It is their interaction or the combination of predictors that I wanted to explore.

 

[hlsmith]

Interaction has a particular definition. Did you examine for additive or multiplicative interaction in your original model? You should be able examine for this while controlling for covariates, unless your model is oversaturated in the first place (too many predictors given the size of the smaller outcome groups).

Heads up for future analyses, stepwise regression gets chewed up in the review process. Covariate selection should be dictated via content knowledge. There are many examples of how using an automated modeling process based on criteria may fumble up mediation or interaction, etc. effects. Since it may be dropping main terms, etc.

Keep the questions coming.

 

[LarsG]

Thanks.
I have two predictors that have an additive effect.
So I was thinking I could compare the relative risk of having one or both, but not sure what test I can use to compare them

 

[hlsmith]

You can run a likelihood test with and without the interaction term included in the model.

What relative risk "having one or both" are you referring to. So do you have a 'significant' multiplicative interaction term or an additive effect. Also, getting relative risks from a logit model isn't (or should) just be a switching of things. If your data is not prospective, it is not appropriate to calculate relative risks since risk involves incidents.

 

[LarsG]

Thank you hlsmith.
I am sorry for not being more clear on the terminology, I am still new to this.
I will try to explain better.

I am looking into the probability of outcome A (binary).
predictor X1 (binary)
predictor X2 (continuous).
I changed X2 to binary (over/under cutoff)
I calculated the risk of A in 4 groups (X1 neg+ X2 neg, X1 pos + X2 neg, X1 pos + X2 pos)
The combined risk of X1 and X2 did not equal the sum of risk X1 X2 (much higher risk with the combo).

I made a logistic regression with X1 binary and X2 continuous (and some other predictors).
Results:
X1 significant.
X2 not significant.
interaction term: not significant.

X1 explains most of A and X2 does not contribute much and is excluded from the model using stepwise AIC method.


X2 is the variable we are trying to understand.
We also did a model where we included the X1 and X2 as a combination (4 levels as factor (X1 neg/X2 neg, X1 pos/X2neg, X1pos/X2pos, X1neg/X2pos))
And here it is very clearly the combination of the two that increases the probability of A. With large OR for the pos/pos but not for the other combos.

This is very exploratory because we do not know much about X2. We just think it makes sense that if you have X1 and high X2 it is probably a bad thing.

 

[hlsmith]

Thanks.

Can you draw out the relationships to better ensure an interaction and not mediation. And it is ensured that neither X1 or X2 may be an effect of A. Which you should refer to A as "Y" in the future to minimize confusion. Y is the traditional representation for the dependent variable. You could have an additive interaction, but plain ol' logistic regression isn't gonna show this to you without some additional work - a la RERI. https://catalyst.harvard.edu/docs/biostatsseminar/VanderWeele2012.pdf

What is the prevalence of the outcome variable.

Also, it may help if you provide more details on the study context, such as -> what do X1, X2, and Y represent.

 

[LarsG]

Thanks for the advise and powerpoint.
I have tried calculating RERI and I got a positive value of 1.2, but I will have to take that to our biostatistian to make sure I did it right. I have never tried that before.
X1 and X cannot be an effect of Y as they always come before Y. They are a pressure measurements in the lungs before surgery and Y is a complication to surgery.
If I plot the probability of Y on y-axis and increasing continuous X2 on x-axis for three distinct levels of X1 i get three curves with different slopes (more steep for high level X1). They have the same starting point where X2 = 0.

 

[hlsmith]

Now X1 has three levels. Post your figure if possible, It may need confidence intervals.

You will also need to slap confidence intervals on the RERI if you got that route.