Transition Probability

[jjangub]

Hi.
I have a question about transition probability.
Q) Markov chain on states {0,1,2,3,4,5} has transition probability matrix (which is 6 by 6 matrix).
okay, the matrix given, but I won't write it here.
1) Find all classes from this matrix.
2) Find lim (n goes to infinity) for i = 0,1,2,3,4,5
I have no idea about 1)
for 2), I learned from the class that lim (n goes to infinity) = = sigma (i=0 to infinity) and sigma (i=0 to infinity) = 1

so since lim (n goes to infinity) is equal to , and this equals to sigma (i=0 to 5)

so using sigma (i=0 to 5) and it is given that sigma (i=0 to infinity) = 1,

so = the answer will be 1/6
(in the last row, all the probabilities are evenly 1/6)

please give me the hint or idea about how to do for 1)
and check for 2).
Thank you...

[Masteras]

he means transient classes and positive recurrent in 1 and in 2 meybe you should find the stationary distribution before answering. But i assume you do are not familiar with what i said, right?

[jjangub]

yes...I am not sure...
the prof went through this part really fast without any example...
so I thought I understood the concept and the theorem, but I do not know
how to apply this to actual example...
He said this prob is really easy...I should have done within 5 min...
Isn't the stationary distribution {} i = 0 to infinity?

[Masteras]

the stationary distribtution π is the one that solves this
π=πP. Can you upload the matrix?

[jjangub]

I don't know how to upload matrix, so
1/3 2/3 0 0 0 0
2/3 1/3 0 0 0 0
0 0 1/4 3/4 0 0
0 0 1/5 4/5 0 0
1/4 0 1/4 0 1/4 1/4
1/6 1/6 1/6 1/6 1/6 1/6

this is the matrix which is given...
and isn't the {1,2} and {3,4} are sets containing recurrent states?

[Masteras]

yes they are. you are right and they are irreducible. 5 and 6 are trnasient. Now there is no unique stationary distribution. So you want for i=1,2,3,4,5,6. Well you can use the diagonalization theorem to find for beginning and then apply the limit

[jjangub]

so I tried with
when i = 0, = when n = 1 and
= when n >= 2
so this equal to = 1/6 + sigma(n = 2 to infinity) n(1/6)((1/3)^(n-2))(2/3)
so if you work out this, I get 4/9 (I just followed how my prof. did in class)

when i = 1, = when n = 1 and
= when n >= 2
since is zero, it is equal to 1/6

when i = 3 and i = 5, I get 1/6

when i = 2, = when n = 1 and
= when n >= 2
so this equal to = 1/6 + sigma(n = 2 to infinity) n(1/6)((4/5)^(n-2))(1/5)
so if you work out this, I get 7/40

when i = 4, = when n = 1 and
= when n >= 1
so I work out this, I get 2/9

and since lim(n goes to infinity) = 1/(sigma(n=0 to 5) n) = 120/161

this sounds crazy...but I just tried...
I don't know this is right or not...
I have to do this by tomorrow...can you tell me something really close to answer?
Thank you...
Bye...

[Masteras]

I agree with the rationale for the f caculations. but i do not agree that .
I think states 2, 3 amd 4 and 5 are transient. thus the answer is zero for them. and are not zero.