1. ## Re: Posterior Distribution

But will rejection sampling work in this case? How do I know it will not work?
It will work whenever the support of your target density is a subset of your proposal density. Since now the support of your target density is the whole real line, you also required that the tail of your proposal is thicker.

How am I able to tell that my probablity of acceptance will be low without using any software packages?
You need some sort of numerical integration. Since the area under the density is 1, the constant M tells you the proportion of samples accepted.

2. ## The Following User Says Thank You to BGM For This Useful Post:

MrAnon9 (12-15-2011)

3. ## Re: Posterior Distribution

Could you tell me what is the best method for my question and reasoning? I am totally baffled

Would appreciate it so much!

4. ## Re: Posterior Distribution

I think I worked out that the proposal density does have thicker tails and that m = 0.00004 so the acceptance rate is 1/0.00004 ? :S

5. ## Re: Posterior Distribution

Hi again, sorry for all the questions but what value of M do I want? and if I don't get this type of value, does this mean I should employ MCMC ?

6. ## Re: Posterior Distribution

I guess one last shot at asking for help here :P

I have f(x) < g(x)*M and I have worked out that M is 0.0004 by substituting xhat = log(5/11) into h(x) where h(x) = f(x) / g(x), and also plotted both the target and the proposal density and noticed that the proposal density was a lot "fatter" than the target density, which was thin. But I don't know what this all means, does this mean rejection sampling is working well here?

7. ## Re: Posterior Distribution

I have not check any of your numerical solution here. But if the claim in your last post is correct, then everything is fine. Even the constant M is not optimal (smallest M that larger then the ratio), the algorithm can still work.

8. ## Re: Posterior Distribution

How can i work out my probablity of acceptance? and also how it rejection sampling fine here? I have plotted the two graphs and saw that the density of the proposal is a lot larger and the target is only a small part of it

Also the acceptance rate is 1/M right? 1/0.00004 = A hugee number, what's that mean and how is that plausible?

9. ## Re: Posterior Distribution

Since there is a few posts concern about the rejection sampling here maybe I just gives a detailed example here, using your problem.

Now the posterior density

where is the normalizing constant.

You do not need to know the exact value of if you are just want the rejection sampling algorithm. However if you want to know "theoretically" the exact acceptence rate you may need to do a numerical integration to get this constant. Of course the acceptance rate can also be estimated by the proportion of acceptance in your simulation.

Now suppose your proposal density is a normal density

Now, dropping the all the constants out, note that

Since we want , we require

In addtion when , in this special case we also require

With these constraints on the parameters, we differentiate the ratio with respect to :

In general you can optimize the ratio by choosing the optimal parameters . But anyway let say you have chosen which satisfy the above constraint. Then the first order condition

And you can easily verify that it indeed is the global maximum point of the desired ratio.

Let say your rejection sampling algorithm is to accept when

Then such constant is optimally chosen by

when
as you have calculated above .

Note that in the process we have omit some constants - if we include them back we can calculate the acceptance rate.
If we accept the sample when , then the acceptance rate

Borrowing Dason's code you can also do the simulation and estimate the acceptance rate from R:
Code:
n <- 100000
t <- rnorm(n, 0, 10)
u <- runif(n)
r <- exp(5*t)/(1+exp(t))^16
M <- 5^5*11^11/16^16
id.accept <- (u <= r/M)
samp <- proposals[id.accept]
accept.rate <- mean(id.accept)
print(accept.rate)
P.S.: The constant should be able to numerically integrated in R, but it seems the integrand overflowed at some numbers.