Assume that an ergodic Markov chain has states s1,s2,...,sk. Let Sj^(n) denote the number

Question

Assume that an ergodic Markov chain has states s₁,s₂,…,s_k. Let S_j⁽ⁿ⁾ denote the number of times that the chain is in state s_j in the ﬁrst n steps. Let w denote the ﬁxed probability row vector for this chain. Show that, regardless of the starting state, the expected value of S_j⁽ⁿ⁾ , divided by n, tends to w_j as n →∞. Hint : If the chain starts in state s_i, then the expected value of S_j⁽ⁿ⁾ is given by the expression

$\sum\limits_{h=0}^{n}{p_{ij}^{(h)}} .$

[latex]\sum\limits_{h=0}^{n}{p_{ij}^{(h)}} .[/latex]

Accepted Answer

Assume that the chain is started in state s_i. Let X_j⁽ⁿ⁾ equal 1 if the chain is in state s_i on the nth step and 0 otherwise. Then

S_j⁽ⁿ⁾ = X_j⁽⁰⁾ + X_j⁽¹⁾ + X_j ⁽²⁾ + . . .X_j⁽ⁿ⁾ .

and

E(X_j⁽ⁿ⁾ ) = Pⁿ_ij .

Thus

[latex]E(S^{(n)}_j)= \sum\limits_{h=0}^{n}{p^{(n)}_{ij}}.[/latex]

If now follows then from Exercise 16 that

[latex]\underset{n\rightarrow \infty }{\lim} \frac{E(S^{(n)}_{j})}{n} = w_j.[/latex]

Question 11.5.23: Assume that an ergodic Markov chain has states s1,s2,...,sk.......

This Question has been Answered.

Already have an account?

Login

Related Answered Questions

(Roberts7) A city is divided into 3 areas 1, 2, and 3. It is estimated that amounts u1, u2, and u3 of pollution are emitted each day from these three areas. A fraction qij of the pollution from region i ends up the next day at region j. A fraction qi =1−Σj qij > 0 goes into the atmosphere and ...

Verified Answer:

Consider a random walker who moves on the integers 0, 1, …, N, moving one step to ...

Verified Answer:

In Example 11.11, deﬁne f(i) to be the proportion of G genes in state i. Show that f is ...

Verified Answer:

Consider an absorbing Markov chain with state space S. Let f be a function deﬁned on S ...

Verified Answer:

A highly simpliﬁed game of “Monopoly” is played on a board with four squares as shown in ...

Verified Answer:

Show that, for an ergodic Markov chain (see Theorem 11.16), Σ j mijwj = Σj zjj -1 = K. ...

Verified Answer:

A perpetual craps game goes on at Charley’s. Jones comes into Charley’s on an evening ...

Verified Answer:

An ergodic Markov chain is started in equilibrium (i.e., with initial probability vector ...

Verified Answer:

Show that any ergodic Markov chain with a symmetric transition matrix (i.e., pij = pji) is ...

Verified Answer:

Let P be the transition matrix of an ergodic Markov chain and P^∗ the reverse transition ...

Verified Answer: