Suppose I have a stock that in in each timeperiod gives a return depending on the state s. Assume that the process starts in s_1 and gives the return r(s_1). Then with probability p(s_1) the state change to s_2 such that the return is r(s_2). The horizon is infinite and geometric discounting is used:

\mathbb E [ \sum_{t=0}^\infty \beta^t r(s_t)]


I figured out that if there are only two states so the state simply changes ones the the situation is like a geometric distribution ... you are waiting for the return to change value ... and the expectation can be found ... i think ... by rewriting the sum to get:

\frac{r_1}{1-\beta} + \frac{\beta}{1-\beta} (r_2 - r_1) \beta^\tau

where only $\tau$ is stochastic being the random time where the state change. Then the expectation of \beta^\tau can be found using the moment generating function of the geometric distribution to get something like \frac{p\beta}{1-(1-p)\beta} ....


My question is how do I find the expectation if there are more than two states but finite states. And simulation and valuefunction iteration are non-acceptable procedures due to merely being numerical ... Im looking for an analytical answer ...