Markov limit expectation


TS Contributor
I need to find a limit involving a discrete finite state markov chain with absorbing state.

It is given that:

\(q(x,c) = r(x) + \beta E_{c'\lvert c}[q(x,c')]\)

so my thought are:

\(q(x,c') = r(x) + \beta E_{c''\lvert c'}[q(x,c'')]\)

such that :

\(q(x,c) = r(x) + \beta r(x) + \beta^2 E_{c''\lvert c}[q(x,c'')]\)

and repeating infinitely to get limit \(\frac{r(x)}{1-\beta}\)

but I do not know how to prove this ...

my guess is also that since there is a single absorbing state then:

\(\lim_{n \rightarrow \infty} E_{c^n\lvert c}[q(x,c^n)] = q(x,a)\)

with \(a \) being the "absorbing" value if that makes sense.