markov chains

I request your help in order to know, how can i configure this problem as a continuous markov chain, need to define the main variable, the states, transition rates, and the matrix. I thought that it could be relationed with the independent status of the machines, because if the machine 1 is...

Hi everyone, My first post! :-) I apologize for how long it will be, but please bear with me. I am looking for help in a topic related to Markov chain modeling with covariates. I have a decent knowledge of the underlying theory, but not exactly how to implement some aspects of it in practical...

I'm working my way through a course in stochastic processes but came across this Lemma in my coursebook where they didn't show the working (Lemma 8.5). I'm not familiar with doing proofs and it isn't taught in any of my courses but I can read/understand them, how did they reach the conclusion of...

I am looking for a test for reducing the order of a markov chain with state space {0,1} from k to 0. The chain is observed only once for length T. I have looked at Hoel (1954) "A Test for Markoff Chains A TEST FOR MARKOFF CHAINS". But it only covers sequential reduction, from K to K-1. Any...

This question concerns defining the likelihood of a set of parameters in a rate matrix of an embedded Markov chain. To get a brief idea of the context I will try to explain the problem with a simpler one first. I am working with evolutionary models (see...

N customers circulate in a closed system with two servers. In Server 1 and Server 2, service times are exponential with parameters µ1 and µ2, respectively. Let state K denotes that there are k customers in Stage 1(thus N-k customers in stage 2). I'd like to determine P(K), which is the...

I have an example that is solved on one of the pages of wikipedia related to markov chains. I want to know how the probabilities (in bold) are calculated (please show the steps/formulas) Example: Another example is the dietary habits of a creature who eats only grapes, cheese or lettuce...