+ Reply to Thread
Results 1 to 8 of 8

Thread: Modeling:Rare Event Data

  1. #1
    Points: 1,992, Level: 26
    Level completed: 92%, Points required for next Level: 8

    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Modeling:Rare Event Data



    Hi,
    How can I do logistic regression of Rare event Data? For my data response rate is 3.1%.

    Thanks,
    Stata

  2. #2
    Bhoot
    Points: 1,434, Level: 21
    Level completed: 34%, Points required for next Level: 66

    Posts
    1,759
    Thanks
    40
    Thanked 128 Times in 107 Posts
    Response rate is not too small. You can still do the logistic regression.

    If you want to increase your response rate , you can think of Biased sampling (Over sampling) technique.
    In the long run, we're all dead.

  3. #3
    Points: 1,992, Level: 26
    Level completed: 92%, Points required for next Level: 8

    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Can you pls. send me some good link for Biased Sampling?

    Thanks,
    Stata

  4. #4
    Bhoot
    Points: 1,434, Level: 21
    Level completed: 34%, Points required for next Level: 66

    Posts
    1,759
    Thanks
    40
    Thanked 128 Times in 107 Posts
    The following link may help

    http://www.gotstat.com/tag/oversampling.aspx
    In the long run, we're all dead.

  5. #5
    Points: 1,992, Level: 26
    Level completed: 92%, Points required for next Level: 8

    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Hi,
    Thank you for the link. I saw the example there.But I have two doubts.

    1. I have 231 events in development sample.So after applying under sampling I'm getting around 470 observations.I have around 10 independent variable. My doubt is that will this method able to estimate the parameters with this much little data?

    2. When I am using logistic regression with "Weight-adjusted Model",the result is showing that no beta is significant,But when I'm using the same data with "Offset-adjusted Model", it is giving some betas as significant.This made me totally confused as same data is giving completely different output. Can you please suggest me which method do I need to follow.

    Thanks in advance,

  6. #6
    Bhoot
    Points: 1,434, Level: 21
    Level completed: 34%, Points required for next Level: 66

    Posts
    1,759
    Thanks
    40
    Thanked 128 Times in 107 Posts
    1.Here you are trying to keep the response rate as around 50 % and that is the reason you are short of observations. 470 observations are very less for logistic regression.
    So you can make response rate around 10% .. so that you will have more observation for model development.

    2. There are lot of variation in the Biased sampling procedure in industry. But I see the "Weght adjusted model" is frequently used.

    You can check in the Score card development or credit scoring books for more information.

    I will also check the same from my end.
    In the long run, we're all dead.

  7. #7
    Points: 1,992, Level: 26
    Level completed: 92%, Points required for next Level: 8

    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thank you . I will check credit scoring book.

  8. #8
    Points: 1,992, Level: 26
    Level completed: 92%, Points required for next Level: 8

    Posts
    17
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Hi Vinux,
    Can you please suggest me some papers for rare event multinomial modeling?
    Actually in my data dependent variable has 3 level, and I have 4% observation for First event,73% observation for second event and 23% observation for third event.When I'm doing multinomial modeling with such data set it is overpredicting level 2 ,underpredicting level 3,and not able to predict the level 1.
    Please help me.

    Thanks in advance

+ Reply to Thread

Similar Threads

  1. Probability of event
    By banebt in forum Probability
    Replies: 7
    Last Post: 03-08-2011, 01:06 PM
  2. Kaplan Meier curve for recurrent event data
    By weibull6 in forum Statistics
    Replies: 2
    Last Post: 11-07-2010, 06:37 AM
  3. Event Clustering in an Event Study
    By Christoph in forum Statistical Research
    Replies: 1
    Last Post: 03-09-2009, 08:41 AM
  4. RARE EVENTS -- Poisson / Binomial -- Need Help
    By jchen123456789 in forum Statistics
    Replies: 2
    Last Post: 01-28-2007, 01:57 PM
  5. Event Certainty
    By shangwuvn in forum Probability
    Replies: 1
    Last Post: 03-12-2006, 12:23 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts








Advertise on Talk Stats