4 Some useful texts

At increasing levels of mathematical sophistication:

• Häggström (2002) “Finite Markov chains and algorithmic applications.”
  Delightful introduction to finite state-space discrete-time Markov chains, from point of view of computer algorithms.

• Grimmett and Stirzaker (2001) “Probability and random processes.”
  Standard undergraduate text on mathematical probability. If you are going to buy one book on probability, this is a good choice because it contains so much material.

• Norris (1998) “Markov chains.”
  Markov chains at a more graduate level of sophistication, revealing what I have concealed, namely the full gory story about \(Q\)-matrices.

• Williams (1991) “Probability with martingales.”
  Excellent graduate text for theory of martingales: mathematically demanding.

4.1 Free on the web

• Doyle and Snell (1984) “Random walks and electric networks”
  Available on the web at http://arxiv.org/abs/math/0001057.
  Lays out (in simple and accessible terms) an important approach to Markov chains using relationship to resistance in electrical networks.

• Kindermann and Snell (1980) “Markov random fields and their applications”
  Available on the web at http://www.ams.org/online_bks/conm1/
  Sublimely accessible treatment of Markov random fields (Markov property, but in space not time).

• Meyn and Tweedie (1993) “Markov chains and stochastic stability”
  Available on the web at http://probability.ca/MT/.
  The place to go if you need to get informed about theoretical results on rates of convergence for Markov chains (e.g. because you are doing MCMC).

• Aldous and Fill (2001) “Reversible Markov Chains and Random Walks on Graphs”
  Only available on the web at http://www.stat.berkeley.edu/~aldous/RWG/book.html
  The best unfinished book on Markov chains known to us.

4.2 Going deeper

• Kingman (1993) “Poisson processes.”
  Very good introduction to the wide circle of ideas surrounding the Poisson process.

• Kelly (1979) “Reversibility and stochastic networks.”
  We’ll cover reversibility briefly in the lectures, but this shows just how powerful the technique is.

• Lindvall (1992) “Lectures on the coupling method.”
  We’ll also talk briefly about the beautiful concept of coupling for Markov chains; this book gives a very nice introduction.

• Steele (2004) “The Cauchy-Schwarz master class.”
  The book to read if you decide you need to know more about (mathematical) inequality.

• Aldous (1989) “Probability approximations via the Poisson clumping heuristic.”
  See www.stat.berkeley.edu/~aldous/Research/research80.html.
  A book full of what ought to be true; hence good for stimulating research problems and also for ways of computing heuristic answers.

• Øksendal (2003) “Stochastic differential equations.”
  An accessible introduction to Brownian motion and stochastic calculus, which we do not cover at all.

• Stoyan, Kendall, and Mecke (1987) “Stochastic geometry and its applications.”
  Discusses a range of techniques used to handle probability in geometric contexts.

References

Aldous, David J. 1989. Probability Approximations via the Poisson Clumping Heuristic. Vol. 77. Applied Mathematical Sciences. New York: Springer-Verlag.

Aldous, David J., and James A. Fill. 2001. Reversible Markov Chains and Random Walks on Graphs. Unpublished. http://www.stat.berkeley.edu/~aldous/RWG/book.html.

Doyle, Peter G., and J. Laurie Snell. 1984. Random Walks and Electric Networks. Vol. 22. Carus Mathematical Monographs. Washington, DC: Mathematical Association of America.

Grimmett, Geoffrey R., and David R. Stirzaker. 2001. Probability and Random Processes. Third. New York: Oxford University Press.

Häggström, Olle. 2002. Finite Markov Chains and Algorithmic Applications. Vol. 52. London Mathematical Society Student Texts. Cambridge: Cambridge University Press.

Kelly, Frank P. 1979. Reversibility and Stochastic Networks. Chichester: John Wiley & Sons Ltd.

Kindermann, Ross, and J. Laurie Snell. 1980. Markov Random Fields and Their Applications. Vol. 1. Contemporary Mathematics. Providence, R.I.: American Mathematical Society. http://www.ams.org/online_bks/conm1/.

Kingman, J. F. C. 1993. Poisson Processes. Vol. 3. Oxford Studies in Probability. New York: The Clarendon Press Oxford University Press.

Lindvall, Torgny. 1992. Lectures on the coupling method. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. New York: John Wiley & Sons Inc.

Meyn, S. P., and R. L. Tweedie. 1993. Markov Chains and Stochastic Stability. Communications and Control Engineering Series. London: Springer-Verlag London Ltd. http://probability.ca/MT/.

Norris, J. R. 1998. Markov Chains. Vol. 2. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge University Press.

Steele, J. Michael. 2004. The Cauchy-Schwarz Master Class. MAA Problem Books Series. Washington, DC: Mathematical Association of America.

Stoyan, D., W. S. Kendall, and J. Mecke. 1987. Stochastic Geometry and Its Applications. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. Chichester: John Wiley & Sons Ltd.

Williams, David. 1991. Probability with Martingales. Cambridge Mathematical Textbooks. Cambridge: Cambridge University Press.

Øksendal, Bernt. 2003. Stochastic Differential Equations. Sixth. Universitext. Berlin: Springer-Verlag.