+ Reply to Thread
Results 1 to 2 of 2

Thread: Calculating a stochastic matrix with multiple states

  1. #1
    Points: 4, Level: 1
    Level completed: 7%, Points required for next Level: 46

    Thanked 0 Times in 0 Posts

    Calculating a stochastic matrix with multiple states

    I am struggling with how to calculate the values of a Markov matrix which has multiple states.

    For example,

    Imagine an unfair 6 sided dice. The chance of rolling a 1,2,3,4,5 or 6 is

    0.3, 0.25, 0.2, 0.12, 0.10, 0.03 respectively

    The idea is that you keep rolling the dice until the accumulated total is greater than 6. That gives us 7 states, 1,2,3,4,5,6,7+

    This gives me the following transition matrix

    0.3, 0

    0.25, ?

    0.2, ?





    The first column in the Markov matrix is simply the probabilities listed above along with 0 for the chance of rolling greater than a 6 on first throw. The next column obviously starts with 0 because it is not possible to still have an accumulated chance of 1 after 2 throws. After is where I get confused. I have no idea how to calculate the probability of being in state 2 after the second throw because this relies on current state being a 1 and then throwing another 1 (surely it isn't simply 0.3). Even more confusing to me is the probability of being in state 3. This means the current state either has to be 1 and roll a 2 or current state 2 and roll a 1.

    If anyone could explain to me how to calculate the probability of state 2 in column 2 and state 3 in column 2 I would greatly appreciate it.

    Thanks for your time.

  2. #2
    TS Contributor
    Points: 22,410, Level: 93
    Level completed: 6%, Points required for next Level: 940

    Thanked 565 Times in 537 Posts

    Re: Calculating a stochastic matrix with multiple states

    I think first of all you need clarify what you want.

    Transition matrix is a matrix containing transition probabilities - which are conditional probabilities: given the previous state, the probability of transiting to the next state.

    So if you are constructing a transition matrix for this problem, most likely you will have the 7 states you described, and the last state 7+ is the absorption state. With that in mind, you will soon form a upper triangular matrix. If you want to find out the number of steps lead to absorption, then you are calculating the nth power of the transition matrix.

    And from your example, it seems that you are trying to figure out the probability mass function of the sum of i.i.d. dice throws.

+ Reply to Thread


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Advertise on Talk Stats