Markov chain Monte Carlo methods have revolutionized mathematical computation and enabled statistical inference within many previously intractable models. In this context, Hamiltonian dynamics have been proposed as an efficient way of building chains that can explore probability densities efficiently. The method emerges from physics and geometry, and these links have been extensively studied over the past thirty years. The aim of this review is to provide a comprehensive introduction to the geometric tools used in Hamiltonian Monte Carlo at a level accessible to statisticians, machine learners, and other users of the methodology with only a basic understanding of Monte Carlo methods. This will be complemented with some discussion of the most recent advances in the field, which we believe will become increasingly relevant to scientists.


Article metrics loading...

Loading full text...

Full text loading...


Literature Cited

  1. Abdulle A, Vilmart G, Zygalakis KC. 2015. Long time accuracy of Lie-Trotter splitting methods for Langevin dynamics. SIAM J. Numer. Anal. 53:1–16 [Google Scholar]
  2. Ajay A, Walters W, Murcko M. 1998. Can we learn to distinguish between “drug-like” and “nondrug-like” molecules?. J. Medic. Chem. 41:3314–24 [Google Scholar]
  3. Akhmatskaya E, Reich S. 2008. GSHMC: an efficient method for molecular simulation. J. Comp. Phys. 227:4934–54 [Google Scholar]
  4. Amari SI. 1987. Differential Geometrical Methods in Statistics New York: Springer
  5. Arnold V. 1989. Mathematical Methods of Classical Mechanics New York: Springer
  6. Barndorff-Nielsen OE, Cox DR, Reid N. 1986. The role of differential geometry in statistical theory. Int. Stat. Rev. 54:83–96 [Google Scholar]
  7. Berne BJ, Straub JE. 1997. Novel methods of sampling phase space in the simulation of biological systems. Curr. Opin. Struct. Biol. 7:181–89 [Google Scholar]
  8. Beskos A, Girolami M, Lan S, Farrell PE, Stuart AM. 2016. Geometric MCMC for infinite-dimensional inverse problems. arXiv1606.06351 [stat.CO]
  9. Beskos A, Pinski FJ, Sanz-Serna JM, Stuart AM. 2011. Hybrid Monte Carlo on Hilbert spaces. Stoch. Proc. Appl. 121:2201–30 [Google Scholar]
  10. Beskos A, Roberts GO, Sanz-Serna JM, Stuart AM. 2013. Optimal tuning of the hybrid Monte-Carlo algorithm. Bernoulli 19:1501–34 [Google Scholar]
  11. Betancourt MJ. 2015. The fundamental incompatibility of Hamiltonian Monte Carlo and data subsampling. In Proceedings of the 37th International Conference on Machine Learning (ICML 37). ed. F Bach, D Blei, pp. 533–40. Brookline, MA: Microtome
  12. Betancourt MJ. 2017. A conceptual introduction to Hamiltonian Monte Carlo. arXiv1701.02434 [stat.ME]
  13. Betancourt MJ, Byrne S, Livingstone S, Girolami M. 2016. The geometric foundations of Hamiltonian Monte Carlo. Bernoulli 23:2257–98 [Google Scholar]
  14. Blanes S, Casas F, Sanz-Serna JM. 2014. Numerical integrators for the hybrid Monte Carlo method. SIAM J. Sci. Comput. 36:1752–69 [Google Scholar]
  15. Bui-Thanh T, Girolami M. 2014. Solving large-scale PDE-constrained Bayesian inverse problems with Riemann manifold Hamiltonian Monte Carlo. Inverse Probl 30:114014 [Google Scholar]
  16. Byrne S, Girolami M. 2013. Geodesic Monte Carlo on embedded manifolds. Scand. J. Stat. 40:825–45 [Google Scholar]
  17. Campos CM, Sanz-Serna JM. 2015. Extra chance generalized hybrid Monte Carlo. J. Comput. Phys. 281:365–74 [Google Scholar]
  18. Campostrini M, Rossi P. 1990. A comparison of numerical algorithms for dynamical fermions. Nucl. Phys. B 329:753–64 [Google Scholar]
  19. Carpenter B, Gelman A, Hoffman M, Lee D, Goodrich B. et al. 2016. Stan: a probabilistic programming language. J. Stat. Softw. 76:1–32 [Google Scholar]
  20. Chen C, Ding N, Carin L. 2015. On the convergence of stochastic gradient MCMC algorithms with high-order integrators. Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS 15) C Cortes, DD Lee, M Sugiyama, R Garnett 2278–86 Cambridge, MA: MIT Press [Google Scholar]
  21. Chen T, Fox EB, Guestrin C. 2014. Stochastic gradient Hamiltonian Monte Carlo. Proceedings of the 31st International Conference on Machine Learning (ICML 2014) EP Xing, T Jebara 3663–76 Brookline, MA: Microtome [Google Scholar]
  22. Cheung SH, Beck JL. 2009. Simulation with application to structural dynamic models with many uncertain parameters. J. Eng. Mech. 135:243–55 [Google Scholar]
  23. Chiu SN, Stoyan D, Kendall W, Mecke J. 2013. Stochastic Geometry and Its Applications New York: Wiley
  24. Choo K, Fleet DJ. 2001. People tracking using hybrid Monte Carlo filtering. Proc. 8th IEEE Int. Conf. Computer Vis.321–28 New York: IEEE [Google Scholar]
  25. Cotter SL, Roberts GO, Stuart A, White D. 2013. MCMC methods for functions: modifying old algorithms to make them faster. Stat. Sci. 28:424–46 [Google Scholar]
  26. Critchley F, Marriott P, Salmon M. 1993. Preferred point geometry and statistical manifolds. Ann. Stat. 21:1197–224 [Google Scholar]
  27. Diaconis P, Holmes S, Neal RM. 2000. Analysis of a nonreversible Markov chain sampler. Ann. Appl. Prob. 10:726–52 [Google Scholar]
  28. Dryden IL, Kent JT. 2015. Geometry Driven Statistics New York: Wiley
  29. Dryden IL, Mardia KV. 1998. Statistical Shape Analysis New York: Wiley
  30. Duane S, Kennedy AD, Pendleton BJ, Roweth D. 1987. Hybrid Monte Carlo. Phys. Lett. B 195:216–22 [Google Scholar]
  31. Efron B. 1982. Defining the curvature of a statistical problem (with applications to second order efficiency). Ann. Stat. 10:1100–20 [Google Scholar]
  32. Fang Y, Sanz-Serna JM, Skeel RD. 2014. Compressible generalized hybrid Monte Carlo. J. Chem. Phys. 140:174108 [Google Scholar]
  33. Fernández-Pendás M, Escribano B, Radivojević T, Akhmatskaya E. 2014. Constant pressure hybrid Monte Carlo simulations in GROMACS. J. Mol. Model. 20:2487 [Google Scholar]
  34. Frankel T. 2012. The Geometry of Physics Cambridge, UK: Cambridge Univ. Press. , 3rd ed..
  35. Fredrickson GH, Ganesan V, Drolet F. 2002. Field-theoretical computer simulation methods for polymer and complex fluids. Macromolecules 35:16–39 [Google Scholar]
  36. Gelfand AE, Smith AFM. 1990. Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc. 85:398–409 [Google Scholar]
  37. Girolami M, Calderhead B. 2011. Riemann manifold Langevin and Hamiltonian Monte Carlo methods. J. R. Stat. Soc. B 73:123–214 [Google Scholar]
  38. Hairer E, Lubich C, Wanner G. 2006. Geometric Numerical Integration Algorithms for Ordinary Differential Equations New York: Springer
  39. Hansmann UH, Okamoto Y. 1999. New Monte Carlo algorithms for protein folding. Curr. Opin. Struct. Biol. 9:177–83 [Google Scholar]
  40. Hastings WK. 1970. Monte Carlo sampling methods using Markov Chains and their applications. Biometrika 57:97–109 [Google Scholar]
  41. Hoffman M, Gelman A. 2014. The no-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res. 15:1593–623 [Google Scholar]
  42. Horowitz AM. 1991. A generalized guided Monte Carlo algorithm. Phys. Lett. B 268:247–52 [Google Scholar]
  43. Izaguirre JA, Hampton SS. 2004. Shadow hybrid Monte Carlo: an efficient propagator in phase space of macromolecules. J. Comput. Phys. 200:581–604 [Google Scholar]
  44. Kass RE, Vos PW. 1997. Geometrical Foundations of Asymptotic Inference New York: Wiley
  45. Kennedy AD, Silva PJ, Clark MA. 2012. Shadow Hamiltonians, Poisson brackets, and gauge theories. Phys. Rev. D 87:034511 [Google Scholar]
  46. Konukoglu E, Relan J, Cilingir U, Menze BH, Chinchapatnam P. et al. 2011. Efficient probabilistic model personalization integrating uncertainty on data and parameters: application to eikonal-diffusion models in cardiac electrophysiology. Prog. Biophys. Mol. Biol. 107:134–46 [Google Scholar]
  47. Kramer A, Calderhead B, Radde N. 2014. Hamiltonian Monte Carlo methods for efficient parameter estimation in steady state dynamical systems. BMC Bioinform 15:253 [Google Scholar]
  48. Lan S, Bui-Thanh T, Christie M, Girolami M. 2016. Emulation of higher-order tensors in manifold Monte Carlo methods for Bayesian inverse problems. J. Comput. Phys. 308:81–101 [Google Scholar]
  49. Lan S, Stathopoulos V, Shahbaba B, Girolami M. 2015. Markov chain Monte Carlo from Lagrangian dynamics. J. Comput. Graphic. Stat. 24:357–78 [Google Scholar]
  50. Lan S, Streets J, Shahbaba B. 2014. Wormhole Hamiltonian Monte Carlo. Proc. 28th AAAI Conf. Artif. Intell.1953–59 Palo Alto, CA: AAAI [Google Scholar]
  51. Landau DP, Binder K. 2009. A Guide to Monte-Carlo Simulations in Statistical Physics Cambridge, UK: Cambridge Univ. Press
  52. Ledoux M. 2001. The Concentration of Measure Phenomenon Providence, RI: Am. Math. Soc.
  53. Leimkuhler B, Reich S. 2004. Simulating Hamiltonian Dynamics Cambridge, UK: Cambridge Univ. Press
  54. Livingstone S, Betancourt M, Byrne S, Girolami M. 2015. On the geometric ergodicity of Hamiltonian Monte Carlo. arXiv1601.08057 [stat.CO]
  55. Livingstone S, Girolami M. 2014. Information-geometric Markov chain Monte Carlo methods using diffusions. Entropy 16:3074–102 [Google Scholar]
  56. Ma Y, Chen T, Fox EB. 2015. A complete recipe for stochastic gradient MCMC. Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS 15) C Cortes, ND Lawrence, DD Lee, M Sugiyama, R Garnett 2917–25 Cambridge, MA: MIT Press [Google Scholar]
  57. MacKay DJC. 2003. Information Theory, Inference, and Learning Algorithms Cambridge, UK: Cambridge Univ. Press
  58. Mehlig B, Heermann D, Forrest B. 1992. Hybrid Monte Carlo method for condensed-matter systems. Phys. Rev. B 45:679–85 [Google Scholar]
  59. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E. 1953. Equation of state calculations by fast computing machines. J. Chem. Phys. 21:1087–92 [Google Scholar]
  60. Meyn S, Tweedie R. 1993. Markov Chains and Stochastic Stability Berlin: Springer-Verlag
  61. Murray MK, Rice JW. 1993. Differential Geometry and Statistics New York: Springer
  62. Neal RM. 2011. MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo S Brooks, A Gelman, GL Jones, XL Meng 113–62 Boca Raton, FL: Chapman and Hall/CRC [Google Scholar]
  63. Nishimura A, Dunson D. 2016. Geometrically tempered Hamiltonian Monte Carlo.. arXiv1604.00872 [stat.CO]
  64. Rao CR. 1945. Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 37:81–91 [Google Scholar]
  65. Robert C, Casella G. 2004. Monte Carlo Statistical Methods New York: Springer
  66. Robert C, Casella G. 2011. A short history of Markov chain Monte Carlo: subjective recollections from incomplete data. Stat. Sci. 26:102–15 [Google Scholar]
  67. Roberts GO, Rosenthal JS. 1998. Optimal scaling of discrete approximations to Langevin diffusions. J. R. Stat. Soc. B 60:255–68 [Google Scholar]
  68. Rossky PJ, Doll JD, Friedman HL. 1978. Brownian dynamics as smart Monte Carlo simulation. J. Chem. Phys. 69:4628–33 [Google Scholar]
  69. Scalettar R, Scalapino DJ, Sugar RL. 1986. New algorithm for the numerical simulation of fermions. Phys. Rev. B 34:7911–17 [Google Scholar]
  70. Schroeder KB, McElreath R, Nettle D. 2013. Variants at serotonin transporter and 2A receptor genes predict cooperative behavior differentially according to presence of punishment. PNAS 110:3955–60 [Google Scholar]
  71. Sen MK, Biswas R. 2017. Transdimensional seismic inversion using the reversible jump Hamiltonian Monte Carlo algorithm. Geophysics 82:119–34 [Google Scholar]
  72. Skeel RD, Hardy DJ. 2001. Practical construction of modified Hamiltonians. SIAM J. Sci. Comput. 23:1172–88 [Google Scholar]
  73. Stuart AM. 2010. Inverse problems: a Bayesian perspective. Acta Numer 19:451–559 [Google Scholar]
  74. Sweet CR, Hampton SS, Skeel RD, Izaguirre JA. 2009. A separable shadow Hamiltonian hybrid Monte Carlo method. J. Chem. Phys. 131:174106 [Google Scholar]
  75. Teh YW, Thiery AH, Vollmer S. 2014. Consistency and fluctuations for stochastic gradient Langevin dynamics. arXiv1409.0578 [stat.ML]
  76. Vollmer SJ, Zygalakis KC, Teh YW. 2015. (Non-) asymptotic properties of stochastic gradient Langevin dynamics. arXiv1501.00438 [stat.ME]
  77. Wang Z, Mohamed S, de Freitas N. 2013. Adaptive Hamiltonian and Riemann manifold Monte Carlo samplers. Proceedings of the 30th International Conference on Machine Learning (ICML 2013) S Dasgupta, D McAllester 1462–70 Brookline, MA: Microtome [Google Scholar]
  78. Welling M, Teh YW. 2011. Bayesian learning via stochastic gradient Langevin dynamics. Proceedings of the 28th International Conference on International Conference on Machine Learning (ICML 2011)681–88 Madison, WI: Omnipress [Google Scholar]
  79. Yoshida H. 1990. Construction of higher order symplectic integrators. Phys. Lett. A 150:262–68 [Google Scholar]

Data & Media loading...

Supplemental Material

Supplementary Data

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error