+ Reply to Thread
Page 1 of 3 1 2 3 LastLast
Results 1 to 15 of 38

Thread: Posterior Distribution

  1. #1
    Points: 1,278, Level: 19
    Level completed: 78%, Points required for next Level: 22

    Posts
    135
    Thanks
    13
    Thanked 0 Times in 0 Posts

    Posterior Distribution




    Suppose I have X_{1}..X_{n} which are i.i.d as Bernoulli random variables with P(X_{i} = 1) = \frac{e^\theta}{1+e^\theta} and the prior is normal with mean 0 and variance 100. How can I find the posterior distribution if the number of X_{i} equal to one is 5 and n is 16?

    What do I use for the likelihood here? Do I differentiate \frac{e^\theta}{1+e^\theta} since it's CDF of logistic distribution or do I use the pdf of a bernoulli?

    The question confuses me
    Last edited by MrAnon9; 12-13-2011 at 01:55 PM.

  2. #2
    Devorador de queso
    Points: 95,983, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,938
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Posterior Distribution

    If I told you P(Xi = 1) = p then what would the likelihood be? Ok now substitute \frac{e^\theta}{1 + e^\theta} in for p and you have your likelihood.

  3. #3
    Points: 1,278, Level: 19
    Level completed: 78%, Points required for next Level: 22

    Posts
    135
    Thanks
    13
    Thanked 0 Times in 0 Posts

    Re: Posterior Distribution

    so I just have to find the product of the 1 over 1 + e ?

  4. #4
    Devorador de queso
    Points: 95,983, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,938
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Posterior Distribution

    What? The likelihood would be a binomial distribution: (16 choose 5) * p^5 (1-p)^11 and replace p with the quantity above.

  5. #5
    Devorador de queso
    Points: 95,983, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,938
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Posterior Distribution

    Also are you sure the prior has a variance of 0?

  6. #6
    Points: 1,278, Level: 19
    Level completed: 78%, Points required for next Level: 22

    Posts
    135
    Thanks
    13
    Thanked 0 Times in 0 Posts

    Re: Posterior Distribution

    I meant 100 sorry

  7. #7
    TS Contributor
    Points: 22,410, Level: 93
    Level completed: 6%, Points required for next Level: 940

    Posts
    3,020
    Thanks
    12
    Thanked 565 Times in 537 Posts

    Re: Posterior Distribution

    Just want to say that a prior with variance 0 is very stubborn - it will not change.

  8. #8
    Devorador de queso
    Points: 95,983, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,938
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Posterior Distribution

    Quote Originally Posted by BGM View Post
    Just want to say that a prior with variance 0 is very stubborn - it will not change.
    I myself am a fan of point mass priors. It makes analyzing the posterior very easy.

  9. #9
    TS Contributor
    Points: 22,410, Level: 93
    Level completed: 6%, Points required for next Level: 940

    Posts
    3,020
    Thanks
    12
    Thanked 565 Times in 537 Posts

    Re: Posterior Distribution

    If you have a point mass prior, you are "not Bayesian" :P

    Anyway back to OP problem, since they are not in a conjugate class, so after using the Bayes theorem to write down the posterior, not much can be simplified.

  10. #10
    Devorador de queso
    Points: 95,983, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,938
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Posterior Distribution

    Quote Originally Posted by BGM View Post
    If you have a point mass prior, you are "not Bayesian" :P
    Why not? You're just a very confident Bayesian. We do it all the time when we actually set what we think some of the hyperparameters are - this is equivalent to giving them a point mass prior.

  11. #11
    Points: 1,278, Level: 19
    Level completed: 78%, Points required for next Level: 22

    Posts
    135
    Thanks
    13
    Thanked 0 Times in 0 Posts

    Re: Posterior Distribution

    Why is it a binomial by the way? Also, can I not get rid of the 16 choose 5 as a constant of proportionality~?

  12. #12
    TS Contributor
    Points: 22,410, Level: 93
    Level completed: 6%, Points required for next Level: 940

    Posts
    3,020
    Thanks
    12
    Thanked 565 Times in 537 Posts

    Re: Posterior Distribution

    The combinatoric coefficient in the front is not important. You will cancel it anyway. You can also think the sample condition on the parameter is just Bernoulli.

  13. #13
    Points: 1,278, Level: 19
    Level completed: 78%, Points required for next Level: 22

    Posts
    135
    Thanks
    13
    Thanked 0 Times in 0 Posts

    Re: Posterior Distribution

    If Bernoulli has PMF \theta^{X_{i}} (1-\theta)^{1-X_{i}} and I find the product of this to get the likelihood function of theta, then where does N choose Xi come into it?

  14. #14
    Devorador de queso
    Points: 95,983, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,938
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Posterior Distribution

    Because you don't know the order that the successes came. There is only 1 way to order having no successes. There are 16 ways to have 1 success... it's the same argument as in the development of the binomial distribution. Mainly because it is a binomial distribution.

  15. #15
    Points: 1,278, Level: 19
    Level completed: 78%, Points required for next Level: 22

    Posts
    135
    Thanks
    13
    Thanked 0 Times in 0 Posts

    Re: Posterior Distribution


    Hi again, I have found the posterior to be e^{5\theta - \frac{\theta^2}{200}} (1+e^{\theta})^{-16}

    Is this correct? And also, now I need to use use a normal prior as the proposal density and describe a rejection sampler for sampling from this posterior density, any ideas on how to do that?
    Last edited by MrAnon9; 12-14-2011 at 04:16 PM.

+ Reply to Thread
Page 1 of 3 1 2 3 LastLast

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats