## Markov probability and stock return

Suppose I have a stock that in in each timeperiod gives a return depending on the state s. Assume that the process starts in and gives the return . Then with probability p(s_1) the state change to such that the return is . The horizon is infinite and geometric discounting is used:

I figured out that if there are only two states so the state simply changes ones the the situation is like a geometric distribution ... you are waiting for the return to change value ... and the expectation can be found ... i think ... by rewriting the sum to get:

where only $\tau$ is stochastic being the random time where the state change. Then the expectation of can be found using the moment generating function of the geometric distribution to get something like ....

My question is how do I find the expectation if there are more than two states but finite states. And simulation and valuefunction iteration are non-acceptable procedures due to merely being numerical ... Im looking for an analytical answer ...