Markov chain Monte Carlo (MCMC) algorithms are an indispensable tool for performing Bayesian inference. This review discusses widely used sampling algorithms and illustrates their implementation on a probit regression model for lupus data. The examples considered highlight the importance of tuning the simulation parameters and underscore the important contributions of modern developments such as adaptive MCMC. We then use the theory underlying MCMC to explain the validity of the algorithms considered and to assess the variance of the resulting Monte Carlo estimators.


Article metrics loading...

Loading full text...

Full text loading...


Literature Cited

  1. Adler SL. 1981. Over-relaxation methods for the Monte Carlo evaluation of the partition function for multiquadratic actions. Phys. Rev. D 23:2901–4 [Google Scholar]
  2. Albert JH, Chib S. 1993. Bayesian analysis of binary and polychotomous response data. J. Am. Stat. Assoc. 88:669–79 [Google Scholar]
  3. Amit Y. 1991. On rates of convergence of stochastic relaxation for Gaussian and non-Gaussian distributions. J. Multivar. Anal. 38:82–100 [Google Scholar]
  4. Amit Y. 1996. Convergence properties of the Gibbs sampler for perturbations of Gaussians. Ann. Stat. 24:122–40 [Google Scholar]
  5. Andrieu C, Moulines E, Priouret P. 2005. Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optim. 44:283–312 [Google Scholar]
  6. Bai Y, Craiu RV, DiNarzo AF. 2011. Divide and conquer: a mixture-based approach to regional adaptation for MCMC. J. Comput. Graph. Stat. 20:63–79 [Google Scholar]
  7. Barone P, Frigessi A. 1990. Improving stochastic relaxation for Gaussian random fields. Probab. Eng. Inf. Sci. 4:369–89 [Google Scholar]
  8. Bedard M. 2006. On the robustness of optimal scaling for random walk Metropolis algorithms PhD Thesis, Department of Statistics, Univ. Toronto [Google Scholar]
  9. Brooks S, Gelman A, Jones GL, Meng X-L. 2011. Handbook of Markov Chain Monte Carlo Boca Raton, FL: Chapman & Hall/CRC [Google Scholar]
  10. Brooks SP, Gelman A. 1998. General methods for monitoring convergence of iterative simulations. J. Comput. Graph. Stat. 7:434–55 [Google Scholar]
  11. Casarin R, Craiu RV, Leisen F. 2013. Interacting multiple try algorithms with different proposal distributions. Stat. Comput. 23:185–200 [Google Scholar]
  12. Chen M-H, Shao Q-M, Ibrahim JG. 2000. Monte Carlo Methods in Bayesian Computation New York: Springer [Google Scholar]
  13. Craiu RV, Lemieux C. 2007. Acceleration of the multiple-try Metropolis algorithm using antithetic and stratified sampling. Stat. Comput. 17:109–20 [Google Scholar]
  14. Craiu RV, Meng X-L. 2005. Multi-process parallel antithetic coupling for forward and backward MCMC. Ann. Stat. 33:661–97 [Google Scholar]
  15. Craiu RV, Meng X-L. 2011. Perfection within reach: exact MCMC sampling. Handbook of Markov Chain Monte Carlo S Brooks, A Gelman, GL Jones, X-L Meng 199–226 Boca Raton, FL: Chapman & Hall/CRC [Google Scholar]
  16. Craiu RV, Rosenthal JS, Yang C. 2009. Learn from thy neighbor: parallel-chain adaptive and regional MCMC. J. Am. Stat. Assoc. 104:1454–66 [Google Scholar]
  17. Douc R, Moulines E, Rosenthal JS. 2004. Quantitative bounds on convergence of time-inhomogeneous Markov chains. Ann. Appl. Probab. 14:1643–65 [Google Scholar]
  18. Flegal JM, Haran M, Jones GL. 2008. Markov chain Monte Carlo: Can we trust the third significant figure?. Stat. Sci. 23:250–60 [Google Scholar]
  19. Gelfand AE, Smith AFM. 1990. Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc. 85:398–409 [Google Scholar]
  20. Gelman A, Rubin DB. 1992. Inference from iterative simulation using multiple sequences. Stat. Sci. 7:457–72 [Google Scholar]
  21. Geman S, Geman D. 1984. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6:721–41 [Google Scholar]
  22. Geyer CJ. 1992. Practical Markov chain Monte Carlo (with discussion). Stat. Sci. 7:473–83 [Google Scholar]
  23. Green PJ. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82:711–32 [Google Scholar]
  24. Green PJ, Mira A. 2001. Delayed rejection in reversible jump Metropolis-Hastings. Biometrika 88:1035–53 [Google Scholar]
  25. Haario H, Saksman E, Tamminen J. 2001. An adaptive Metropolis algorithm. Bernoulli 7:223–42 [Google Scholar]
  26. Hastings WK. 1970. Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57:97–109 [Google Scholar]
  27. Jones G, Hobert J. 2001. Honest exploration of intractable probability distributions via Markov chain Monte Carlo. Stat. Sci. 16:312–34 [Google Scholar]
  28. Liu JS. 2001. Monte Carlo Strategies in Scientific Computing New York: Springer [Google Scholar]
  29. Liu JS, Liang F, Wong WH. 2000. The multiple-try method and local optimization in Metropolis sampling. J. Am. Stat. Assoc. 95:121–34 [Google Scholar]
  30. Liu JS, Wong WH, Kong A. 1994. Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes. Biometrika 81:27–40 [Google Scholar]
  31. Liu JS, Wong WH, Kong A. 1995. Covariance structure and convergence rate of the Gibbs sampler with various scans. J. R. Stat. Soc. B 57:157–69 [Google Scholar]
  32. Liu JS, Wu YN. 1999. Parameter expansion for data augmentation. J. Am. Stat. Assoc. 94:1264–74 [Google Scholar]
  33. Meng X-L, van Dyk DA. 1999. Seeking efficient data augmentation schemes via conditional and marginal augmentation. Biometrika 86:301–20 [Google Scholar]
  34. Mengersen KL, Tweedie RL. 1996. Rates of convergence of the Hastings and Metropolis algorithms. Ann. Stat. 24:101–21 [Google Scholar]
  35. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E. 1953. Equations of state calculations by fast computing machines. J. Chem. Phys. 21:1087–92 [Google Scholar]
  36. Meyn SP, Tweedie RL. 1993. Markov Chains and Stochastic Stability London: Springer-Verlag [Google Scholar]
  37. Neal RM. 1995. Suppressing random walks in Markov chain Monte Carlo using ordered overrelaxation Tech. Rep. 9508, Univ. Toronto. Dep. Stat., Toronto, Can. http://arxiv.org/pdf/bayes-an/9506004v1.pdf [Google Scholar]
  38. Papaspiliopoulos O, Roberts GO. 2008. Stability of the Gibbs sampler for Bayesian hierarchical models. Ann. Stat. 36:95–117 [Google Scholar]
  39. Propp JG, Wilson DB. 1996. Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Struct. Algorithms 9:223–52 [Google Scholar]
  40. Richardson S, Green PJ. 1997. On Bayesian analysis of mixtures with an unknown number of components (with discussion). J. R. Stat. Soc. B 59:731–92 [Google Scholar]
  41. Robert CP, Casella G. 2004. Monte Carlo Statistical Methods New York: Springer [Google Scholar]
  42. Robert CP, Casella G. 2010. Introducing Monte Carlo Methods with R New York: Springer [Google Scholar]
  43. Roberts GO, Gelman A, Gilks WR. 1997. Weak convergence and optimal scaling of random walk Metropolis algorithms. Ann. Appl. Probab. 7:110–20 [Google Scholar]
  44. Roberts GO, Rosenthal JS. 1997. Geometric ergodicity and hybrid Markov chains. Electron. Commun. Probab. 2:213–25 [Google Scholar]
  45. Roberts GO, Rosenthal JS. 2001. Optimal scaling for various Metropolis-Hastings algorithms. Stat. Sci. 16:351–67 [Google Scholar]
  46. Roberts GO, Rosenthal JS. 2004. General state space Markov chains and MCMC algorithms. Probab. Surv. 1:20–71 [Google Scholar]
  47. Roberts GO, Rosenthal JS. 2007. Coupling and ergodicity of adaptive Markov chain Monte Carlo algorithms. J. Appl. Probab. 44:458–75 [Google Scholar]
  48. Roberts GO, Rosenthal JS. 2009. Examples of adaptive MCMC. J. Comput. Graph. Stat. 18:349–67 [Google Scholar]
  49. Roberts GO, Tweedie RL. 1996. Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms. Biometrika 83:95–110 [Google Scholar]
  50. Rosenthal JS. 1995. Minorization conditions and convergence rates for Markov chain Monte Carlo. J. Am. Stat. Assoc. 90:558–66 [Google Scholar]
  51. Rosenthal JS. 2001. A review of asymptotic convergence for general state space Markov chains. Far East J. Theor. Stat. 5:37–50 [Google Scholar]
  52. Rosenthal JS. 2002. Quantitative convergence rates of Markov chains: a simple account. Electron. Commun. Probab. 7:123–28 [Google Scholar]
  53. Rosenthal JS, Roberts GO. 2011. Quantitative non-geometric convergence bounds for independence samplers. Methodol. Comput. Appl. Probab. 13:391–403 [Google Scholar]
  54. Spiegelhalter DJ, Best NG, Carlin BP, Van Der Linde A. 2002. Bayesian measures of model complexity and fit (with discussion). J. R. Stat. Soc. B 64:583–639 [Google Scholar]
  55. Tanner MA, Wong WH. 1987. The calculation of posterior distributions by data augmentation. J. Am. Stat. Assoc. 82:528–40 [Google Scholar]
  56. Tierney L. 1994. Markov chains for exploring posterior distributions. Ann. Stat. 22:1701–28 [Google Scholar]
  57. van Dyk DA, Meng X-L. 2001. The art of data augmentation (with discussion). J. Comput. Graph. Stat. 10:1–111 [Google Scholar]

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error