1. ## Transition Probability Matrix

The 6 faces of a fair die are marked with the numbers 1,2,3,4,5,6. We roll the die repeatedly and independently, noting the numbers that come up. We say that Xn = j if j is the largest number that has come up in the first n tosses. (Define X0=1). The resulting DTMC has state space S={1,2,3,4,5,6}. Give its transition probability matrix.

So I understand the basics of how to set up a transition probability matrix, but for some reason this one is confusing me. Any help on setting it up would be great! Thank you!

2. ## Re: Transition Probability Matrix

The transition matrix is

\begin{bmatrix}
\frac {1} {6} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6}
& \frac {1} {6} \\[0.3em]
0 & \frac {1} {3} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6}
& \frac {1} {6} \\[0.3em]
0 & 0 & \frac {1} {2} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6} \\[0.3em]
0 & 0 & 0 & \frac {2} {3} & \frac {1} {6} & \frac {1} {6} \\[0.3em]
0 & 0 & 0 & 0 & \frac {5} {6} & \frac {1} {6} \\[0.3em]
0 & 0 & 0 & 0 & 0 & 1\\[0.3em]
\end{bmatrix}

Sorry cannot type it out [LaTeX Error: String is too long (469, limit 400)]

Anyway the point is that the maximum process
is monotonic increasing, and it increase only when the new roll is greater,
Otherwise, it will stay at the old value.

3. ## Re: Transition Probability Matrix

That helps so much thank you!

4. ## Re: Transition Probability Matrix

Originally Posted by BGM
The transition matrix is

\begin{bmatrix}
\frac {1} {6} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6}
& \frac {1} {6} \\[0.3em]
0 & \frac {1} {3} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6}
& \frac {1} {6} \\[0.3em]
0 & 0 & \frac {1} {2} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6} \\[0.3em]
0 & 0 & 0 & \frac {2} {3} & \frac {1} {6} & \frac {1} {6} \\[0.3em]
0 & 0 & 0 & 0 & \frac {5} {6} & \frac {1} {6} \\[0.3em]
0 & 0 & 0 & 0 & 0 & 1\\[0.3em]
\end{bmatrix}

Sorry cannot type it out [LaTeX Error: String is too long (469, limit 400)]

$\begin{bmatrix}\frac {1} {6} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6}& \frac {1} {6} \\[0.3em]0 & \frac {1} {3} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6}& \frac {1} {6} \\[0.3em]0 & 0 & \frac {1} {2} & \frac {1} {6} & \frac {1} {6} & \frac {1} {6} \\[0.3em]0 & 0 & 0 & \frac {2} {3} & \frac {1} {6} & \frac {1} {6} \\[0.3em]0 & 0 & 0 & 0 & \frac {5} {6} & \frac {1} {6} \\[0.3em]0 & 0 & 0 & 0 & 0 & 1\\[0.3em]\end{bmatrix}$

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts