Posterior Distribution

#1
Suppose I have \(X_{1}..X_{n}\) which are i.i.d as Bernoulli random variables with \(P(X_{i} = 1) = \frac{e^\theta}{1+e^\theta}\) and the prior is normal with mean 0 and variance 100. How can I find the posterior distribution if the number of \(X_{i}\) equal to one is 5 and n is 16?

What do I use for the likelihood here? Do I differentiate \(\frac{e^\theta}{1+e^\theta}\) since it's CDF of logistic distribution or do I use the pdf of a bernoulli?

The question confuses me :(
 
Last edited:

Dason

Ambassador to the humans
#2
If I told you P(Xi = 1) = p then what would the likelihood be? Ok now substitute \(\frac{e^\theta}{1 + e^\theta}\) in for p and you have your likelihood.
 

Dason

Ambassador to the humans
#4
What? The likelihood would be a binomial distribution: (16 choose 5) * p^5 (1-p)^11 and replace p with the quantity above.
 

BGM

TS Contributor
#9
If you have a point mass prior, you are "not Bayesian" :p

Anyway back to OP problem, since they are not in a conjugate class, so after using the Bayes theorem to write down the posterior, not much can be simplified.
 

Dason

Ambassador to the humans
#10
If you have a point mass prior, you are "not Bayesian" :p
Why not? You're just a very confident Bayesian. We do it all the time when we actually set what we think some of the hyperparameters are - this is equivalent to giving them a point mass prior.
 

BGM

TS Contributor
#12
The combinatoric coefficient in the front is not important. You will cancel it anyway. You can also think the sample condition on the parameter is just Bernoulli.
 
#13
If Bernoulli has PMF \(\theta^{X_{i}} (1-\theta)^{1-X_{i}}\) and I find the product of this to get the likelihood function of theta, then where does N choose Xi come into it?
 

Dason

Ambassador to the humans
#14
Because you don't know the order that the successes came. There is only 1 way to order having no successes. There are 16 ways to have 1 success... it's the same argument as in the development of the binomial distribution. Mainly because it is a binomial distribution.
 
#15
Hi again, I have found the posterior to be \(e^{5\theta - \frac{\theta^2}{200}} (1+e^{\theta})^{-16}\)

Is this correct? And also, now I need to use use a normal prior as the proposal density and describe a rejection sampler for sampling from this posterior density, any ideas on how to do that?
 
Last edited:

BGM

TS Contributor
#17
Yes, you are correct to say the posterior pdf is proportional to the function you have found.

Have you learn the rejection sampling?
 

Dason

Ambassador to the humans
#20
You can use whatever way to get a sample from the posterior you want. Unless it's for an assignment and your teacher is requiring you to obtain the sample a certain way.