1.

If a Markov-chain is irreducible, aperiodic and and positive recurrent we can use the theorem to find the limit probabilities. And we get that

[math]\lim_{n \to \infty} P^{n}_{ij}=\lim_{n \to \infty} P^{n}_{jj}=\pi_{j}[/math].

A book says that [math] \pi_{j} [/math] gives the mean time the markov chain is in state j in the long run. but how do we prove this? I mean, how can they interpret a limiting probability, as the mean time a markov-chain is in a given state?

2. Can the theorem be used if we do not have finite states? In one book it says that the theorem works when the chain is irreducible and ergodic(and ergodic = positive recurrent and aperiodic). But another book does not use positive recurrent, just recurrent?