+ Reply to Thread
Results 1 to 3 of 3

Thread: Confused: Metropolis-Hastings, its variations and the Hastings correction ratio

  1. #1
    Points: 40, Level: 1
    Level completed: 80%, Points required for next Level: 10

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Confused: Metropolis-Hastings, its variations and the Hastings correction ratio




    Hello everyone,

    Over the past few weeks I have been studying MCMC and Metropolis-Hastings algorithms. Unfortunately I am stuck and confused with the different variations of the algorithm when I try to practically implement it with code. I was looking for some code examples online but the more I look the more confused I become. Codes that are supposed to implement Metropolis-Hastings implement Random-Walk Metropolis or Independent Metropolis-Hastings. Also most of the times people are silently skipping the practical implementation of the Hastings Correction Ratio because they are sampling from a gaussian distribution. Other times they do implement it but it does not look correct to me because it does not equal to 1 (yes all of the examples I found use a gaussian proposal distribution so it should be equal to 1 right?). So, I would be more than grateful if someone could give me code simple examples (in R or C or python or whatever you prefer) of the following:

    (1) The standard Non-Random-Walk Metropolis Hastings.
    (2) Non-Random-Walk Independent Metropolis Hastings.
    (3) Random-Walk Metropolis Hastings.
    Both should calculate the Hastings Correction Ratio even if you use a gaussian/uniform/symmetric distribution. Please don't just copy-paste websites that partially answer my question because most likely I have already seen them. Also, could someone confirm that the following are different algorithms and not the exact same algorithm with a different name: Metropolis-Hastings, Independent Metropolis Hastings, Random Walk Metropolis Hastings, Metropolis, Independent Metropolis, Random Walk Metropolis.

    Thank you very much for your time.

    EDIT:
    One of the things that confuses me the most is that according to the bibliography the Hastings Correction Ratio when using a normal proposal distribution must be equal to one, but in ALL the examples I find online that is not the case.
    eg:
    http://www.lancs.ac.uk/~jamest/Group/stats4.html
    http://www.mas.ncl.ac.uk/~ndjw1/teac...rop/indep.html
    Am I misreading the bibliography?
    Last edited by Bekos; 07-19-2013 at 08:24 PM.

  2. #2
    Points: 40, Level: 1
    Level completed: 80%, Points required for next Level: 10

    Posts
    2
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Confused: Metropolis-Hastings, its variations and the Hastings correction ratio

    Things are a bit more clear now. I have manage to figure out how to implement Random-Walk M-H and Independent M-H. Turns out that the proposal ratio of a gaussian in the independent M-H does not have to be equal to 1. I haven't figure out how to implement the Vanilla M-H though. The algorithms says that at the start of each iteration I have to "draw a sample from the proposal distribution taking into consideration the current sample". All the implementations I found on the internet do the following: They re-center the gaussian proposal at the current sample and then they randomly draw a sample. This turns out to be a Random-Walk M-H. I can not figure out how I can implement an M-H with a gaussian proposal distribution without making it Random-Walk!

  3. #3
    Points: 3, Level: 1
    Level completed: 5%, Points required for next Level: 47

    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Question Re: Confused: Metropolis-Hastings, its variations and the Hastings correction ratio


    Quote Originally Posted by Bekos View Post
    Things are a bit more clear now. I have manage to figure out how to implement Random-Walk M-H and Independent M-H. Turns out that the proposal ratio of a gaussian in the independent M-H does not have to be equal to 1. I haven't figure out how to implement the Vanilla M-H though. The algorithms says that at the start of each iteration I have to "draw a sample from the proposal distribution taking into consideration the current sample". All the implementations I found on the internet do the following: They re-center the gaussian proposal at the current sample and then they randomly draw a sample. This turns out to be a Random-Walk M-H. I can not figure out how I can implement an M-H with a gaussian proposal distribution without making it Random-Walk!
    You mean Proposal ratio or Correction ratio?

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats