Odds ratio of 1 for bivariate logistic regression

[noetsi]

My wonderous memory is that an odds ratio near 1 is not remotely substantively important, much like a zero slope would be in OLS. But I have a highly significant p value and while I know several thousand cases is a lot, a p under .0001 and a Odds ratio of 0 makes little sense to me. Any comments on this?

The variable should be signficant (the one with an odds ratio near 1) but obviously is not. My guess given the data is that non-linearity is involved (there are no other variables in this model). Anyone know how you do non-linear modeling in logistic regression?

[Jake]

How "near" to 1 is the odds ratio exactly? As a more general comment, some concrete output would help us evaluate this.

[Dason]

My wonderous memory is that an odds ratio near 1 is not remotely substantively important, much like a zero slope would be in OLS. But I have a highly significant p value and while I know several thousand cases is a lot, a p under .0001 and a Odds ratio of 0 makes little sense to me. Any comments on this?

If the odds ratio is near 0 wouldn't you expect it to be significant?

The variable should be signficant (the one with an odds ratio near 1) but obviously is not. My guess given the data is that non-linearity is involved (there are no other variables in this model). Anyone know how you do non-linear modeling in logistic regression?

Add higher orders terms? Use a quadratic term instead of just a linear term.

You don't tell us much about your data - what is the predictor like? Is it continuous? Do you have repeated observations for any values of the predictor? I wrote up some code a while to check whether the linearity assumption is reasonable for logistic regression (in R) and I could post that if you're interested.

[noetsi]

I made a mistake Dason in saying that the odds ratio was near o. I meant near 1.

My data is several thousand cases (my SAS is defunct for a week so I can't pull up the exact numbers) where someone either is successfully rehabilitated or not (this is the dependent variable). The independent variable is how many days they were in the system before their case was closed (a continuous variable). My odds ratio is .99. My actualy slope is almost 0 (I know interpreting it is dangerous).

I found this comments on one site, but I have never seen this before so I don't know if this is an unusual opinion or not.

Note that odds ratios for continuous independent variables tend to be close to one, this does NOT suggest that the coefficients are insignificant.

http://www.appstate.edu/~whiteheadjc...ogit/intro.htm

[Dason]

It could very well be a sample size issue. Just like how it's easy to find a significant difference between two means even if the observed difference is close to 0 if you have a large enough sample size - it's easy enough to find a significant slope even if it's small if you have a large enough sample size.

It sounds like you probably have a lot of observations for each value of the independent variable (is the predictor actually discrete?). If this is the case you could fairly easily explore this graphically to see if the linearity assumption is reasonable.

[noetsi]

Your first sentence is what I thought Dason. That the p value was significant despite a low effect size (meaning the odds ratio) simply because we have a lot of cases. But then I read the comment above (which I can not find anywhere else) which suggests a value very close to 1 in the odds ratio did not mean a weak effect when the independent variable was quantitative. That only the p mattered in this case.


Do you agree with that?

The predictor is the number of days someone was in the system so I would call it discrete, It is in days so I dont know if there are many predicted value for each IV value (that is there might not be a lot of people who stayed in 99 days or 120 days for example). One thing to add to the confusion is that looking at discriptive statistics those who are successfully rehabed had an median of 700 days and those who did not had about 600 days. Which seems like quite a lot.

[Jake]

For testing linearity, if it were me I'd probably just try adding some higher-order polynomial terms and see if they significantly help model fit. But another way of checking it out would be to divide the predictor into 10 (or some similar number of) equally-spaced bins, take the average of the 0/1 responses within each bin, apply the logit transformation to these proportions, and plot them. The plotted logits should be at least approximately linear.

[Dason]

For testing linearity, if it were me I'd probably just try adding some higher-order polynomial terms and see if they significantly help model fit. But another way of checking it out would be to divide the predictor into 10 (or some similar number of) equally-spaced bins, take the average of the 0/1 responses within each bin, apply the logit transformation to these proportions, and plot them. The plotted logits should be at least approximately linear.

That second suggestion is essentially the code I was talking about. I go about it several different ways but the way you mention is included in the list. I definitely think it's a good idea to check that out. Just adding higher order terms you might miss something that you might catch by looking at it graphically.

[noetsi]

I would like the code, if I can think of the way to get the data into r....

[noetsi]

I tested for non-linearity by running month, a quadratic term, and a cubic term. I had 18000 cases. My p value for the estimates of the quadratic and cubic term was signficant, but the odds ratio for the cubic term was 1 and it was 1.01 for the quadratic term. The AIC was lower with the cubic and quadratic terms (23321.68 compared to 23495)

[Jake]

You should test the quadratic and cubic terms using separate models. That is, test the quadratic term in a model that does not include the cubic term; and then test the cubic in a model that also includes the quadratic term, but ignore the quadratic term in this model. This is because only the highest-order term is invariant to the scaling of the predictor. To see this, note that the quadratic term basically tests whether the second derivative differs from 0. In a model that only goes up to a quadratic term, the second derivative is the same for all values of the predictor, so we don't have to worry about things like where the 0-point of the predictor is. But in a model that also includes a cubic term, the second derivative is different at different values of the predictor--it is now the third derivative, represented by the cubic term, that is invariant.

[noetsi]

Ok. I have some question about that.

If I do this and I find that 1) the terms are signficant but have an odds ratio of 0, is it reasonable to argue there is substantive non-linearity. I have 18000 cases and I am comming up with signficant p's for the cubic and quadratic terms, but with the odds ratio being 1 (or in one case .999).

When I run the quadratic, and then the quadratic and cubic term I include the linear term correct? And if both the quadratic and cubic term are signficant in their individual model, do I then run a combined model with cubic and quadratic and interpret both from taht model (I understand you don't do that originally)?

Is there any rule of thumb of how much AIC increase has to occur to be a meaningful increase (as with a F change test for example in OLS).

[Dason]

Depending on what you're trying to do any difference could be "meaningful" if it's significant. Also are these odds ratios based on the unit you actually care about (which I believe you said was months) or was it based on "days"?

We can't tell you if an odds ratio is meaningful for you or not. The tests will tell you if it's significant - but without knowing more about the data it's hard to say whether or not the effect size is meaningful. One thing you could do is compared the predicted probabilities for the different models and see how different they actually are for various days of interests. So compare what the model predicts for 30 days using just a linear term versus a quadratic or cubic. Compare the predictions for 2 years (if this is a reasonable input). See what differences it makes. I would say it's a lot easier to compare whether or not the difference in the predicted probabilities is "meaningful" than comparing whether or not some odds ratio for a higher order term is meaningful. (Especially since trying to interpret odds ratios for higher order terms is... well... not so straightforward)