Question 11.5.19: Show that, for an ergodic Markov chain (see Theorem 11.16), ......

Show that, for an ergodic Markov chain (see Theorem 11.16),

\sum\limits_{j}{m_{ij}w_j} = \sum\limits_{j}{z_{jj}-1}= K.

The second expression above shows that the number K is independent of i. The number K is called Kemeny’s constant. A prize was offered to the first person to give an intuitively plausible reason for the above sum to be independent of i. (See also Exercise 24.)