Fwd: Markov chain solution

Begin forwarded message: