# markov chains

1. ### how can i set this problem as a continuous markov chain?

I request your help in order to know, how can i configure this problem as a continuous markov chain, need to define the main variable, the states, transition rates, and the matrix. I thought that it could be relationed with the independent status of the machines, because if the machine 1 is...
2. ### Markov chain with covariates

Hi everyone, My first post! :-) I apologize for how long it will be, but please bear with me. I am looking for help in a topic related to Markov chain modeling with covariates. I have a decent knowledge of the underlying theory, but not exactly how to implement some aspects of it in practical...
3. ### Prove: Birth-death chain has stationary distribution -> satisfy detail bal. equations

I'm working my way through a course in stochastic processes but came across this Lemma in my coursebook where they didn't show the working (Lemma 8.5). I'm not familiar with doing proofs and it isn't taught in any of my courses but I can read/understand them, how did they reach the conclusion of...
4. ### Looking for litterature on Markov chains

I am looking for a test for reducing the order of a markov chain with state space {0,1} from k to 0. The chain is observed only once for length T. I have looked at Hoel (1954) "A Test for Markoff Chains A TEST FOR MARKOFF CHAINS". But it only covers sequential reduction, from K to K-1. Any...
5. ### [Likelihood of embedded Markov chain] Likelihood of Goldman Yang (GY) rate matrix

This question concerns defining the likelihood of a set of parameters in a rate matrix of an embedded Markov chain. To get a brief idea of the context I will try to explain the problem with a simpler one first. I am working with evolutionary models (see...
6. ### customer service problem

N customers circulate in a closed system with two servers. In Server 1 and Server 2, service times are exponential with parameters µ1 and µ2, respectively. Let state K denotes that there are k customers in Stage 1(thus N-k customers in stage 2). I'd like to determine P(K), which is the...
7. ### Markov Chain probabilities

I have an example that is solved on one of the pages of wikipedia related to markov chains. I want to know how the probabilities (in bold) are calculated (please show the steps/formulas) Example: Another example is the dietary habits of a creature who eats only grapes, cheese or lettuce...