# Thread: Logistic regression convergence issues

1. ## Logistic regression convergence issues

Hi Folks!

I'm using logistic regression for classifying records into groups. The issue with logistic regression is that if any cell in the contigency table of dependent v/s independent equals 0 then there are convergence issues. Usually the program warns with messages like "complete separation/ quasi- complete separation" etc. and there are no unique maximum likelihood solutions in this case.

To explain what I mean by contingency table of DV v/s IV, if the equation is Logit(Y)=a+X1+X2+... then frequency table of Y v/s X1, Y v/s X2 and so on. Say if any of the cells of Y v/s X1 is 0 then algorithm won't converge. If I remove X1 from analysis then it converges but unfortunately, I can't remove these variables.

My questions are as follows:
1) Does anyone have any idea if Discriminant analysis also suffers from this problem?

2) Any alternatives to solving this problem other than logistic regression and discriminant analysis? The solution should be well implemented by the softwares.

3) Even if there is no unique solution, the program still produces estimates based on the last iteration. In this case, are odds ratios or even parameter estimates robust? By robust I mean can I safely draw conclusions on them expecting that the unique solution would be approximately close to this.

Your positive response is deeply appreciated.

2. ## Re: Logistic regression convergence issues

Sorry, it might have been a bit late answer. Although I am not intending to answer your 3 questions specifically, I want to share my thoughts.
I have suffered similar problems in the past when there were 0 counts in some of the groups, which resulted in massive odds ratio and large standard errors. I got around the problem by combing some of the levels of the factor so that each level of factor had a non zero count.

3. ## Re: Logistic regression convergence issues

Gelman has a paper (LINK) on logistic regression with weakly informative priors, which will purportedly provide sensible parameter estimates even in the face of complete separation. The paper includes example code for fitting such models using the bayesglm() function from the "arm" package in R. I haven't worked through the paper or played with the function, so I can't say much beyond this.

4. ## The Following User Says Thank You to Jake For This Useful Post:

jrai (04-04-2012)

5. ## Re: Logistic regression convergence issues

Yay! Bayesian analysis FTW.

6. ## Re: Logistic regression convergence issues

I thought that might make you happy.

7. ## Re: Logistic regression convergence issues

To be honest my first thought when I saw the OP was a Bayesian approach. I probably wouldn't have taken exactly the same approach as Gelman but I would trust Gelman's advice more than mine on this type of issue. But I believe jrai is mainly a SAS person and I don't know if SAS has that capability so I skuttled away from that option for a while. Although it does have proc MCMC so you could program it yourself...

8. ## The Following User Says Thank You to Dason For This Useful Post:

jrai (04-04-2012)

9. ## Re: Logistic regression convergence issues

SAS seem to have the Bayesian version of most of the PROCs.

For eg: PROC BGENMOD, PROC BLIFEREG, PROC BPHREG.
http://support.sas.com/rnd/app/da/bay******.html

10. ## The Following User Says Thank You to ledzep For This Useful Post:

jrai (04-04-2012)

11. ## Re: Logistic regression convergence issues

Well look at that. I guess I don't pay too much attention to SAS news.

12. ## Re: Logistic regression convergence issues

Originally Posted by ledzep
Sorry, it might have been a bit late answer. Although I am not intending to answer your 3 questions specifically, I want to share my thoughts.
I have suffered similar problems in the past when there were 0 counts in some of the groups, which resulted in massive odds ratio and large standard errors. I got around the problem by combing some of the levels of the factor so that each level of factor had a non zero count.
Thanks for the idea. I thought about this but it doesn't work for the client though. Well I found that the results for variables with non-zero counts are robust. I compared the results from model with all the variables with those from the model with variables with non-zero count and interestingly the results are very close. I don't know whether this is generally true or not but for the current case it seems to be true.

 Tweet

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts