+ Reply to Thread
Results 1 to 6 of 6

Thread: Model convergence and fit

  1. #1
    Points: 682, Level: 13
    Level completed: 64%, Points required for next Level: 18

    Posts
    20
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Model convergence and fit




    Hi, I am not understanding what the difference is between model convergence (as in a multivariable logistic regression model), versus the fit of a model.

    In SAS, does QIC tell us the fit of a model?

  2. #2
    Points: 1,741, Level: 24
    Level completed: 41%, Points required for next Level: 59

    Posts
    230
    Thanks
    37
    Thanked 68 Times in 59 Posts

    Re: Model convergence and fit

    Quote Originally Posted by martink View Post
    Hi, I am not understanding what the difference is between model convergence (as in a multivariable logistic regression model), versus the fit of a model.

    In SAS, does QIC tell us the fit of a model?
    In very loose terms (you can definitely ask if you'd like a more technical answer):

    Model Convergence- the model is estimated in a way that uses multiple runs (think successive approximations). If a solution is available, the model will converge on a solution. If, for one reason or another, the model cannot converge, then it means that an "appropriate" solution was not found during the estimation procedure. Be cautious, though, because some programs will still give you model output with an error message for nonconvergence (the solutions aren't appropriate, so you need to figure out what's going on).

    Model Fit- the degree of how "compatible" the model is with the data (maybe thought of as how well the model explains or predicts the outcome variable).

    QIC is similar to the AIC for model fit-- smaller is better. The QIC, as I understand it (from a text I just referenced), is utilized to help determine the correlation structure with better fit. However, caution should be applied, as the book (Allison's Logitic Regression in SAS) indicates that the QIC can be misleading in that nonsensical correlation structures (per theory or prior knowledge) can appear to fit the data "better". Use subject matter knowledge when using the QIC to evaluate the correlation structure if there is a disagreement between structures.
    Last edited by ondansetron; 01-16-2017 at 06:40 PM.

  3. #3
    Points: 682, Level: 13
    Level completed: 64%, Points required for next Level: 18

    Posts
    20
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Model convergence and fit

    Thanks, that was really helpful!

    One question would be: what can cause a model to not converge? Am I correct in saying that sparsity in data would potentially cause convergence problems since the model is not populated with enough data to work properly/to converge on a solution?

    In terms of AIC to assess model fit, is it only used as a relative measure like QIC i.e. must compare it to AICs from other models, and lowest AIC is the model with the best fit?

    Lastly, I am using proc genmod in SAS for modified Poisson multivariable regression (with a binary outcome), and I heard that model fit cannot be assessed in this scenario. Are you aware of the reason for this?

    Similarly, I heard that the assumptions of proc genmod for modified poisson regression cannot actually be verified using SAS. Do you why this is? Couldn't identify anything related to this in the literature.

  4. #4
    Points: 1,741, Level: 24
    Level completed: 41%, Points required for next Level: 59

    Posts
    230
    Thanks
    37
    Thanked 68 Times in 59 Posts

    Re: Model convergence and fit

    Quote Originally Posted by martink View Post
    Thanks, that was really helpful!

    One question would be: what can cause a model to not converge? Am I correct in saying that sparsity in data would potentially cause convergence problems since the model is not populated with enough data to work properly/to converge on a solution?
    There are quite a few reasons why this might occur, and data sparsity could be an issue (at least, in logistic regression). Complete and quasi-complete separation of data would also be another reason. I would imagine something similar holds for Poisson regression, but I can't be sure because I'm not very familiar with Poisson regression and it's pitfalls. Someone else might be better able to help you with the particular question. If you are getting a message about convergence, try posting the message or log file, and we'll try to figure it out.

    Quote Originally Posted by martink View Post
    In terms of AIC to assess model fit, is it only used as a relative measure like QIC i.e. must compare it to AICs from other models, and lowest AIC is the model with the best fit?
    With my limited experience of the QIC, I believe the QIC is used within one model for the correlation structures, but the QICu would be used to compare fit between nested (or non-nested) models. AIC can be used in a similar way for nested and non-nested models.

    Quote Originally Posted by martink View Post
    Lastly, I am using proc genmod in SAS for modified Poisson multivariable regression (with a binary outcome), and I heard that model fit cannot be assessed in this scenario. Are you aware of the reason for this?
    I'm not sure if or why this would be so, but that's probably due to my inexperience with this kind of model.

    Quote Originally Posted by martink View Post
    Similarly, I heard that the assumptions of proc genmod for modified poisson regression cannot actually be verified using SAS. Do you why this is? Couldn't identify anything related to this in the literature.
    Similar to above, I'm not sure due to my inexperience with this kind of modeling. My guess would be that you could test the assumptions, although there is no built in option in SAS. I could be mistaken, though.

    If you have access to a copy of Paul Allison's "Logistic Regression Using the SAS System: Theory and Application" (2nd edition), I would recommend looking through the Poisson chapter. (I haven't had the chance to read it fully as of yet.) The book, however, is rated well and is very conversational (and it uses SAS for it's examples), so this seems like it might be useful for you if you can access it (unless another user is able to clarify these points for you).

    Sorry I couldn't be of further assistance. Convergence issues crop up in different methods that use iterative estimation processes, so I thought I would make an attempt to help out.

  5. #5
    Points: 682, Level: 13
    Level completed: 64%, Points required for next Level: 18

    Posts
    20
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Model convergence and fit

    Thanks again, very helpful info.

    I was wondering if you could describe how the model is estimated as it converges on a solution. What is the mechanism at work behind that estimation process (for modified poisson for instance). Are you referring to MLE for instance?

  6. #6
    Points: 1,741, Level: 24
    Level completed: 41%, Points required for next Level: 59

    Posts
    230
    Thanks
    37
    Thanked 68 Times in 59 Posts

    Re: Model convergence and fit


    Quote Originally Posted by martink View Post
    Thanks again, very helpful info.

    I was wondering if you could describe how the model is estimated as it converges on a solution. What is the mechanism at work behind that estimation process (for modified poisson for instance). Are you referring to MLE for instance?
    I believe you're right about MLE being used for this kind of modeling. You can loosely think of it as a more sophisticated version of trial and error; iteration 1 has some values for the coefficients, then iterations 2 makes adjustments and estimates new coefficients, iteration 3 makes adjustments again...This occurs until the likelihood function has been properly maximized. Convergence occurs when the model estimation is sufficiently close to the actual observations of Y, where sufficiently close is measured by some tiny threshold (often shown as a "Convergence Criterion").

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats