Confused: Metropolis-Hastings, its variations and the Hastings correction ratio

#1
Hello everyone,

Over the past few weeks I have been studying MCMC and Metropolis-Hastings algorithms. Unfortunately I am stuck and confused with the different variations of the algorithm when I try to practically implement it with code. I was looking for some code examples online but the more I look the more confused I become. Codes that are supposed to implement Metropolis-Hastings implement Random-Walk Metropolis or Independent Metropolis-Hastings. Also most of the times people are silently skipping the practical implementation of the Hastings Correction Ratio because they are sampling from a gaussian distribution. Other times they do implement it but it does not look correct to me because it does not equal to 1 (yes all of the examples I found use a gaussian proposal distribution so it should be equal to 1 right?). So, I would be more than grateful if someone could give me code simple examples (in R or C or python or whatever you prefer) of the following:

(1) The standard Non-Random-Walk Metropolis Hastings.
(2) Non-Random-Walk Independent Metropolis Hastings.
(3) Random-Walk Metropolis Hastings.
Both should calculate the Hastings Correction Ratio even if you use a gaussian/uniform/symmetric distribution. Please don't just copy-paste websites that partially answer my question because most likely I have already seen them. Also, could someone confirm that the following are different algorithms and not the exact same algorithm with a different name: Metropolis-Hastings, Independent Metropolis Hastings, Random Walk Metropolis Hastings, Metropolis, Independent Metropolis, Random Walk Metropolis.

Thank you very much for your time.

EDIT:
One of the things that confuses me the most is that according to the bibliography the Hastings Correction Ratio when using a normal proposal distribution must be equal to one, but in ALL the examples I find online that is not the case.
eg:
http://www.lancs.ac.uk/~jamest/Group/stats4.html
http://www.mas.ncl.ac.uk/~ndjw1/teaching/sim/metrop/indep.html
Am I misreading the bibliography?
 
Last edited:
#2
Things are a bit more clear now. I have manage to figure out how to implement Random-Walk M-H and Independent M-H. Turns out that the proposal ratio of a gaussian in the independent M-H does not have to be equal to 1. I haven't figure out how to implement the Vanilla M-H though. The algorithms says that at the start of each iteration I have to "draw a sample from the proposal distribution taking into consideration the current sample". All the implementations I found on the internet do the following: They re-center the gaussian proposal at the current sample and then they randomly draw a sample. This turns out to be a Random-Walk M-H. I can not figure out how I can implement an M-H with a gaussian proposal distribution without making it Random-Walk!
 
#3
Things are a bit more clear now. I have manage to figure out how to implement Random-Walk M-H and Independent M-H. Turns out that the proposal ratio of a gaussian in the independent M-H does not have to be equal to 1. I haven't figure out how to implement the Vanilla M-H though. The algorithms says that at the start of each iteration I have to "draw a sample from the proposal distribution taking into consideration the current sample". All the implementations I found on the internet do the following: They re-center the gaussian proposal at the current sample and then they randomly draw a sample. This turns out to be a Random-Walk M-H. I can not figure out how I can implement an M-H with a gaussian proposal distribution without making it Random-Walk!
You mean Proposal ratio or Correction ratio?