1932

Abstract

Rationality principles such as optimal feedback control and Bayesian inference underpin a probabilistic framework that has accounted for a range of empirical phenomena in biological sensorimotor control. To facilitate the optimization of flexible and robust behaviors consistent with these theories, the ability to construct internal models of the motor system and environmental dynamics can be crucial. In the context of this theoretic formalism, we review the computational roles played by such internal models and the neural and behavioral evidence for their implementation in the brain.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-control-060117-105206
2019-05-03
2024-06-22
Loading full text...

Full text loading...

/deliver/fulltext/control/2/1/annurev-control-060117-105206.html?itemId=/content/journals/10.1146/annurev-control-060117-105206&mimeType=html&fmt=ahah

Literature Cited

  1. 1.  Conant R, Ashby R 1970. Every good regulator of a system must be a model of that system. Int. J. Syst. Sci. 1:89–97
    [Google Scholar]
  2. 2.  Miall RC, Wolpert DM 1996. Forward models for physiological motor control. Neural Netw. 9:1265–79
    [Google Scholar]
  3. 3.  Kawato M 1999. Internal models for motor control and trajectory planning. Curr. Opin. Neurobiol. 9:718–27
    [Google Scholar]
  4. 4.  Craik KJW 1943. The Nature of Explanation Cambridge, UK: Cambridge Univ. Press
    [Google Scholar]
  5. 5.  Popper K 1957. The Poverty of Historicism Boston: Beacon
    [Google Scholar]
  6. 6.  Newell A 1982. The knowledge level. Artif. Intell. 18:87–127
    [Google Scholar]
  7. 7.  MacKay D 2003. Information Theory, Inference, and Learning Algorithms Cambridge, UK: Cambridge Univ. Press
    [Google Scholar]
  8. 8.  Todorov E, Jordan MI 2002. Optimal feedback control as a theory of motor coordination. Nat. Neurosci. 5:1226–35
    [Google Scholar]
  9. 9.  Todorov E 2008. General duality between optimal control and estimation. 2008 47th IEEE Conference on Decision and Control4286–92 New York: IEEE
    [Google Scholar]
  10. 10.  Kappen HJ, Gómez V, Opper M 2012. Optimal control as a graphical model inference problem. Mach. Learn. 87:159–82
    [Google Scholar]
  11. 11.  Todorov E 2004. Optimality principles in sensorimotor control. Nat. Neurosci. 7:907–15
    [Google Scholar]
  12. 12.  Hinton GE, Dayan P, Frey BJ, Neal RM 1995. The “wake-sleep” algorithm for unsupervised neural networks. Science 268:1158–61
    [Google Scholar]
  13. 13.  Faisal A, Selen LPJ, Wolpert DM 2008. Noise in the nervous system. Nat. Rev. Neurosci. 9:292–303
    [Google Scholar]
  14. 14.  Kording K, Ku S, Wolpert D 2004. Bayesian integration in force estimation. J. Neurophysiol. 92:3161–65
    [Google Scholar]
  15. 15.  Kording K, Wolpert D 2004. Bayesian integration in sensorimotor learning. Nature 427:244–47
    [Google Scholar]
  16. 16.  Jazayeri M, Shadlen M 2010. Temporal context calibrates interval timing. Nat. Neurosci. 13:1020–26
    [Google Scholar]
  17. 17.  Tassinari H, Hudson T, Landy M 2006. Combining priors and noisy visual cues in a rapid pointing task. J. Neurosci. 26:10154–63
    [Google Scholar]
  18. 18.  Adams W, Graf E, Ernst M 2004. Experience can change the ‘light-from-above’ prior. Nat. Neurosci. 7:1057–58
    [Google Scholar]
  19. 19.  Kersten D, Mamassian P, Yuille A 2004. Object perception as Bayesian inference. Annu. Rev. Psychol. 55:271–304
    [Google Scholar]
  20. 20.  Stone J, Kerrigan IS, Porrill J 2009. Where is the light? Bayesian perceptual priors for lighting direction. Proc. R. Soc. B 276:1797–804
    [Google Scholar]
  21. 21.  Weiss Y, Simoncelli EP, Adelson EH 2002. Motion illusions as optimal percepts. Nat. Neurosci. 5:598–604
    [Google Scholar]
  22. 22.  Tenenbaum JB, Griffiths TL 2001. Generalization, similarity, and Bayesian inference. Behav. Brain Sci. 24:629–40
    [Google Scholar]
  23. 23.  Griffiths TL, Tenenbaum JB 2009. Theory-based causal induction. Psychol. Rev. 116:661–716
    [Google Scholar]
  24. 24.  Tenenbaum JB, Griffiths TL, Kemp C 2006. Theory-based Bayesian models of inductive learning and reasoning. Trends Cogn. Sci. 10:309–18
    [Google Scholar]
  25. 25.  Houlsby NM, Huszár F, Ghassemi MM, Orbán G, Wolpert DM, Lengyel M 2013. Cognitive tomography reveals complex, task-independent mental representations. Curr. Biol. 23:2169–75
    [Google Scholar]
  26. 26.  Kim J, Greene M, Zlateski A, Lee K, Richardson M et al. 2014. Space-time wiring specificity supports direction selectivity in the retina. Nature 509:331–36
    [Google Scholar]
  27. 27.  Berry MJ II, Brivanlou IH, Jordan TA, Meister M 1999. Anticipation of moving stimuli by the retina. Nature 398:334–38
    [Google Scholar]
  28. 28.  Ma WJ, Beck JM, Latham PE, Pouget A 2006. Bayesian inference with probabilistic population codes. Nat. Neurosci. 9:1432–38
    [Google Scholar]
  29. 29.  Jazayeri M, Movshon JA 2006. Optimal representation of sensory information by neural populations. Nat. Neurosci. 9:690–96
    [Google Scholar]
  30. 30.  Beck JM, Latham PE, Pouget A 2011. Marginalization in neural circuits with divisive normalization. J. Neurosci. 31:15310–19
    [Google Scholar]
  31. 31.  Ganguli D, Simoncelli E 2014. Efficient sensory encoding and Bayesian inference with heterogeneous neural populations. Neural Comput. 26:2103–34
    [Google Scholar]
  32. 32.  Rao RP 2004. Bayesian computation in recurrent neural circuits. Neural Comput. 16:1–38
    [Google Scholar]
  33. 33.  Deneve S 2008. Bayesian spiking neurons I: inference. Neural Comput. 20:91–117
    [Google Scholar]
  34. 34.  Aitchison L, Lengyel M 2016. The Hamiltonian brain: efficient probabilistic inference with excitatory-inhibitory neural circuit dynamics. PLOS Comput. Biol. 12:e1005186
    [Google Scholar]
  35. 35.  Beck JM, Ma WJ, Kiani R, Hanks T, Churchland AK et al. 2008. Probabilistic population codes for Bayesian decision making. Neuron 60:1142–52
    [Google Scholar]
  36. 36.  Yang J, Scholz J, Latash M 2007. The role of kinematic redundancy in adaptation of reaching. Exp. Brain Res. 176:54–69
    [Google Scholar]
  37. 37.  Vogels T, Rajan K, Abbott L 2005. Neural network dynamics. Annu. Rev. Neurosci. 28:357–76
    [Google Scholar]
  38. 38.  Murphy B, Miller K 2009. Balanced amplification: a new mechanism of selective amplification of neural activity patterns. Neuron 61:635–48
    [Google Scholar]
  39. 39.  Denève S, Machens CK 2016. Efficient codes and balanced networks. Nat. Neurosci. 19:375–82
    [Google Scholar]
  40. 40.  Fiser J, Berkes P, Orbán G, Lengyel M 2010. Statistically optimal perception and learning: from behavior to neural representations. Trends Cogn. Sci. 14:119–30
    [Google Scholar]
  41. 41.  Berkes P, Orbán G, Lengyel M, Fiser J 2011. Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science 331:83–87
    [Google Scholar]
  42. 42.  Todorov E, Li W 2003. Optimal control methods suitable for biomechanical systems. Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society 21758–61 New York: IEEE
    [Google Scholar]
  43. 43.  Todorov E 2005. Stochastic optimal control and estimation methods adapted to the noise characteristics of the sensorimotor system. Neural Comput. 17:1084–108
    [Google Scholar]
  44. 44.  Flash T, Hogan N 1985. The coordination of arm movements: an experimentally confirmed mathematical model. J. Neurosci. 5:1688–703
    [Google Scholar]
  45. 45.  Uno Y, Kawato M, Suzuki R 1989. Formation and control of optimal trajectory in human multijoint arm movement. Biol. Cybernet. 61:89–101
    [Google Scholar]
  46. 46.  Bernstein NI 1967. The Coordination and Regulation of Movement Oxford, UK: Pergamon
    [Google Scholar]
  47. 47.  Latash ML, Scholz JP, Schöner G 2002. Motor control strategies revealed in the structure of motor variability. Exerc. Sport Sci. Rev. 30:26–31
    [Google Scholar]
  48. 48.  Diedrichsen J 2007. Optimal task-dependent changes of bimanual feedback control and adaptation. Curr. Biol. 17:1675–79
    [Google Scholar]
  49. 49.  Liu D, Todorov E 2007. Evidence for the flexible sensorimotor strategies predicted by optimal feedback control. J. Neurosci. 27:9354–68
    [Google Scholar]
  50. 50.  Kutch JJ, Kuo AD, Bloch AM, Rymer WZ 2008. Endpoint force fluctuations reveal flexible rather than synergistic patterns of muscle cooperation. J. Neurophysiol. 100:2455–71
    [Google Scholar]
  51. 51.  Harris C, Wolpert D 1998. Signal-dependent noise determines motor planning. Nature 394:780–84
    [Google Scholar]
  52. 52.  Scott SH 2012. The computational and neural basis of voluntary motor control and planning. Trends Cogn. Sci. 16:541–49
    [Google Scholar]
  53. 53.  Diedrichsen J 2007. Optimal task-dependent changes of bimanual feedback control and adaptation. Curr. Biol. 17:1675–79
    [Google Scholar]
  54. 54.  Hatze H, Buys JD 1977. Energy-optimal controls in the mammalian neuromuscular system. Biol. Cybernet. 27:9–20
    [Google Scholar]
  55. 55.  Anderson FC, Pandy MG 2001. Dynamic optimization of human walking. J. Biomech. Eng. 123:381–90
    [Google Scholar]
  56. 56.  Srinivasan M, Ruina A 2006. Computer optimization of a minimal biped model discovers walking and running. Nature 439:72–75
    [Google Scholar]
  57. 57.  Shadmehr R, Huang HJ, Ahmed AA 2016. A representation of effort in decision-making and motor control. Curr. Biol. 26:1929–34
    [Google Scholar]
  58. 58.  Wolpert D, Landy M 2012. Motor control is decision-making. Curr. Opin. Neurobiol. 22:996–1003
    [Google Scholar]
  59. 59.  Trommershauser J, Maloney L, Landy M 2008. Decision making, movement planning and statistical decision theory. Trends Cogn. Sci. 12:291–97
    [Google Scholar]
  60. 60.  Diamond J, Wolpert D, Flanagan J 2017. Rapid target foraging with reach or gaze: The hand looks further ahead than the eye. PLOS Comput. Biol. 13:e1005504
    [Google Scholar]
  61. 61.  Jones K, Hamilton A, Wolpert D 2002. Sources of signal-dependent noise during isometric force production. J. Neurophysiol. 88:1533–44
    [Google Scholar]
  62. 62.  Hamilton A, Jones K, Wolpert D 2004. The scaling of motor noise with muscle strength and motor unit number in humans. Exp. Brain Res. 157:417–30
    [Google Scholar]
  63. 63.  Franklin DW, Burdet E, Tee KP, Osu R, Chew CM et al. 2008. CNS learns stable, accurate, and efficient movements using a simple algorithm. J. Neurosci. 28:11165–73
    [Google Scholar]
  64. 64.  Valero-Cuevas FJ, Venkadesan M, Todorov E 2009. Structured variability of muscle activations supports the minimal intervention principle of motor control. J. Neurophysiol. 102:59–68
    [Google Scholar]
  65. 65.  Nagengast AJ, Braun DA, Wolpert DM 2010. Risk-sensitive optimal feedback control accounts for sensorimotor behavior under uncertainty. PLOS Comput. Biol. 6:e1000857
    [Google Scholar]
  66. 66.  Dimitriou M, Wolpert DM, Franklin DW 2013. The temporal evolution of feedback gains rapidly update to task demands. J. Neurosci. 33:10898–909
    [Google Scholar]
  67. 67.  Yeo SH, Franklin DW, Wolpert DM 2016. When optimal feedback control is not enough: Feedforward strategies are required for optimal control with active sensing. PLOS Comput. Biol. 12:e1005190
    [Google Scholar]
  68. 68.  Scott SH 2016. A functional taxonomy of bottom-up sensory feedback processing for motor actions. Trends Neurosci. 39:512–26
    [Google Scholar]
  69. 69.  Rothwell JC, Traub MM, Marsden CD 1980. Influence of voluntary intent on the human long-latency stretch reflex. Nature 286:496–98
    [Google Scholar]
  70. 70.  Pruszynski JA, Kurtzer I, Nashed JY, Omrani M, Brouwer B, Scott SH 2012. Primary motor cortex underlies multi-joint integration for fast feedback control. Nature 478:387–90
    [Google Scholar]
  71. 71.  Shen L, Alexander GE 1997. Neural correlates of a spatial sensory-to-motor transformation in primary motor cortex. J. Neurophysiol. 77:1171–94
    [Google Scholar]
  72. 72.  Scott S, Kalaska J 1997. Reaching movements with similar hand paths but different arm orientations. I. Activity of individual cells in motor cortex. J. Neurophysiol. 77:826–52
    [Google Scholar]
  73. 73.  Scott SH 2004. Optimal feedback control and the neural basis of volitional motor control. Nat. Rev. Neurosci. 5:532–46
    [Google Scholar]
  74. 74.  Pruszynski J, Scott S 2012. Optimal feedback control and the long-latency stretch response. Exp. Brain Res. 218:341–59
    [Google Scholar]
  75. 75.  McNamee D, Wolpert DM, Lengyel M 2016. Efficient state-space modularization for planning: theory, behavioral and neural signatures. Advances in Neural Information Processing Systems 29 DD Lee, M Sugiyama, UV Luxburg, I Guyon, R Garnett4511–19 Red Hook, NY: Curran
    [Google Scholar]
  76. 76.  Kalman RE 1960. A new approach to linear filtering and prediction problems. J. Basic Eng. 82:35
    [Google Scholar]
  77. 77.  Mitter S, Newton N 2003. A variational approach to nonlinear estimation. SIAM J. Control Optim. 42:1813–33
    [Google Scholar]
  78. 78.  Todorov E 2009. Compositionality of optimal control laws. Advances in Neural Information Processing Systems 22 Y Bengio, D Schuurmans, JD Lafferty, CKI Williams, A Culotta1856–64 Red Hook, NY: Curran
    [Google Scholar]
  79. 79.  Todorov E 2007. Linearly-solvable Markov decision problems. Advances in Neural Information Processing Systems 19 B Schölkopf, JC Platt, T Hoffman1369–76 Red Hook, NY: Curran
    [Google Scholar]
  80. 80.  Rawlik K, Toussaint M, Vijayakumar S 2013. On stochastic optimal control and reinforcement learning by approximate inference (extended abstract). Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence3052–56 Palo Alto, CA: AAAI Press
    [Google Scholar]
  81. 81.  Rao RP, Ballard DH 1999. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2:79–87
    [Google Scholar]
  82. 82.  van Beers RJ 2009. Motor learning is optimally tuned to the properties of motor noise. Neuron 63:406–17
    [Google Scholar]
  83. 83.  Braun DA, Nagengast AJ, Wolpert DM 2011. Risk-sensitivity in sensorimotor control. Front. Hum. Neurosci. 5:1
    [Google Scholar]
  84. 84.  Kawato M, Furukawa K, Suzuki R 1987. A hierarchical neural-network model for control and learning of voluntary movement. Biol. Cybernet. 57:169–85
    [Google Scholar]
  85. 85.  Jordan MI 1996. Computational aspects of motor control and motor learning. Handbook of Perception and Action, Vol. 2: Motor Skills H Heuer, SW Keele71–120 San Diego, CA: Academic
    [Google Scholar]
  86. 86.  von Holst E, Mittelstaedt H 1950. Das Reafferenzprinzip: Wechselwirkungen zwischen Zentralnervensystem und Peripherie. Naturwissenschaften 37:464–76
    [Google Scholar]
  87. 87.  Crapse TB, Sommer MA 2008. Corollary discharge across the animal kingdom. Nat. Rev. Neurosci. 9:587–600
    [Google Scholar]
  88. 88.  Haarmeier T, Bunjes F, Lindner A, Berret E, Thier P 2001. Optimizing visual motion perception during eye movements. Neuron 32:527–35
    [Google Scholar]
  89. 89.  Sommer MA, Wurtz RH 2006. Influence of the thalamus on spatial visual processing in frontal cortex. Nature 444:374–77
    [Google Scholar]
  90. 90.  Bell C 2001. Memory-based expectations in electrosensory systems. Curr. Opin. Neurobiol. 11:481–87
    [Google Scholar]
  91. 91.  Sawtell NB 2017. Neural mechanisms for predicting the sensory consequences of behavior: insights from electrosensory systems. Annu. Rev. Physiol. 79:381–99
    [Google Scholar]
  92. 92.  Rothwell JC, Traub MM, Day BL, Obeso JA, Thomas PK, Marsden CD 1982. Manual motor performance in a deafferented man. Brain 105:515–42
    [Google Scholar]
  93. 93.  Gordon J, Ghilardi MF, Ghez C 1995. Impairments of reaching movements in patients without proprioception. II. Effects of visual information on accuracy. J. Neurophysiol. 73:361–72
    [Google Scholar]
  94. 94.  Wolpert D, Ghahramani Z, Jordan M 1995. An internal model for sensorimotor integration. Science 269:1880–82
    [Google Scholar]
  95. 95.  Diedrichsen J, Verstynen T, Hon A, Lehman SL, Ivry RB 2003. Anticipatory adjustments in the unloading task: Is an efference copy necessary for learning?. Exp. Brain Res. 148:272–76
    [Google Scholar]
  96. 96.  Flanagan JR, Wing AM 1993. Modulation of grip force with load force during point-to-point arm movements. Exp. Brain Res. 95:131–43
    [Google Scholar]
  97. 97.  Kawato M, Kuroda T, Imamizu H, Nakano E, Miyauchi S, Yoshioka T 2003. Internal forward models in the cerebellum: fMRI study on grip force and load force coupling. Prog. Brain Res. 142:171–88
    [Google Scholar]
  98. 98.  Johansson RS, Riso R, Häger C, Bäckström L 1992. Somatosensory control of precision grip during unpredictable pulling loads. I. Changes in load force amplitude. Exp. Brain Res. 89:181–91
    [Google Scholar]
  99. 99.  Crevecoeur F, Scott S 2013. Priors engaged in long-latency responses to mechanical perturbations suggest a rapid update in state estimation. PLOS Comput. Biol. 9:e1003177
    [Google Scholar]
  100. 100.  Crevecoeur F, Munoz DP, Scott SH 2016. Dynamic multisensory integration: Somatosensory speed trumps visual accuracy during feedback control. J. Neurosci. 36:8598–611
    [Google Scholar]
  101. 101.  Nasseroleslami B, Hasson C, Sternad D 2014. Rhythmic manipulation of objects with complex dynamics: predictability over chaos. PLOS Comput. Biol. 10:e1003900
    [Google Scholar]
  102. 102.  Diedrichsen J, Bastian A 2014. Cerebellar function. The Cognitive Neurosciences MS Gazzaniga, GR Mangun451–60 Cambridge, MA: MIT Press. 5th ed.
    [Google Scholar]
  103. 103.  Therrien AS, Bastian AJ 2015. Cerebellar damage impairs internal predictions for sensory and motor function. Curr. Opin. Neurobiol. 33:127–33
    [Google Scholar]
  104. 104.  Ebner TJ, Pasalar S 2008. Cerebellum predicts the future motor state. Cerebellum 7:583–88
    [Google Scholar]
  105. 105.  Hewitt AL, Popa LS, Pasalar S, Hendrix CM, Ebner TJ 2011. Representation of limb kinematics in Purkinje cell simple spike discharge is conserved across multiple tasks. J. Neurophysiol. 106:2232–47
    [Google Scholar]
  106. 106.  Brooks JX, Carriot J, Cullen KE 2015. Learning to expect the unexpected: rapid updating in primate cerebellum during voluntary self-motion. Nat. Neurosci. 18:1310–17
    [Google Scholar]
  107. 107.  Müller F, Dichgans J 1994. Dyscoordination of pinch and lift forces during grasp in patients with cerebellar lesions. Exp. Brain Res. 101:485–92
    [Google Scholar]
  108. 108.  Miall RC, Christensen LOD, Cain O, Stanley J 2007. Disruption of state estimation in the human lateral cerebellum. PLOS Biol. 5:2733–44
    [Google Scholar]
  109. 109.  Bhanpuri NH, Okamura AM, Bastian AJ 2014. Predicting and correcting ataxia using a model of cerebellar function. Brain 137:1931–44
    [Google Scholar]
  110. 110.  Mulliken GH, Musallam S, Andersen RA 2008. Forward estimation of movement state in posterior parietal cortex. PNAS 105:8170–7
    [Google Scholar]
  111. 111.  Buneo CA, Andersen RA 2006. The posterior parietal cortex: sensorimotor interface for the planning and online control of visually guided movements. Neuropsychologia 44:2594–606
    [Google Scholar]
  112. 112.  Desmurget M, Epstein CM, Turner RS, Prablanc C, Alexander GE, Grafton ST 1999. Role of the posterior parietal cortex in updating reaching movements to a visual target. Nat. Neurosci. 2:563–67
    [Google Scholar]
  113. 113.  Desmurget M, Reilly KT, Richard N, Szathmari A, Mottolese C, Sirigu A 2009. Movement intention after parietal cortex stimulation in humans. Science 324:811–13
    [Google Scholar]
  114. 114.  Rushworth MF, Nixon PD, Passingham RE 1997. Parietal cortex and movement. I. Movement selection and reaching. Exp. Brain Res. 117:292–310
    [Google Scholar]
  115. 115.  Sutton R, Barto A 1998. Reinforcement Learning: An Introduction Cambridge, MA: MIT Press
    [Google Scholar]
  116. 116.  Toussaint M, Storkey A 2006. Probabilistic inference for solving discrete and continuous state Markov decision processes. Proceedings of the 23rd International Conference on Machine Learning945–52 New York: ACM
    [Google Scholar]
  117. 117.  Dolan RJ, Dayan P 2013. Goals and habits in the brain. Neuron 80:312–25
    [Google Scholar]
  118. 118.  Schultz W, Dayan P, Montague PR 1997. A neural substrate of prediction and reward. Science 275:1593–99
    [Google Scholar]
  119. 119.  O'Doherty JP, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ 2004. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452–54
    [Google Scholar]
  120. 120.  Kappen HJ, Ruiz HC 2016. Adaptive importance sampling for control and inference. J. Stat. Phys. 162:1244–66
    [Google Scholar]
  121. 121.  Theodorou E, Buchli J, Schaal S 2010. A generalized path integral control approach to reinforcement learning. J. Mach. Learn. Res. 11:3137–81
    [Google Scholar]
  122. 122.  Silver D, Veness J 2010. Monte-Carlo planning in large POMDPs. Advances in Neural Information Processing Systems 23 JD Lafferty, CKI Williams, J Shawe-Taylor, RS Zemel, A Culotta2164–72 Red Hook, NY: Curran
    [Google Scholar]
  123. 123.  Bellman R 1957. Dynamic Programming Princeton, NJ: Princeton Univ. Press
    [Google Scholar]
  124. 124.  D'Avella A, Saltiel P, Bizzi E 2003. Combinations of muscle synergies in the construction of a natural motor behavior. Nat. Neurosci. 6:300–8
    [Google Scholar]
  125. 125.  Huys QJ, Lally N, Faulkner P, Eshel N, Seifritz E et al. 2015. Interplay of approximate planning strategies. PNAS 112:3098–103
    [Google Scholar]
  126. 126.  Botvinick M, Niv Y, Barto A 2009. Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 113:262–80
    [Google Scholar]
  127. 127.  Newell A, Rosenbloom PS 1981. Mechanisms of skill acquisition and the law of practice. Cognitive Skills and Their Acquisition JR Anderson1–56 Cambridge, MA: MIT Press
    [Google Scholar]
  128. 128.  McNamee D, Rangel A, O'Doherty J 2013. Category-dependent and category-independent goal-value codes in human ventromedial prefrontal cortex. Nat. Neurosci. 16:479–85
    [Google Scholar]
  129. 129.  Padoa-Schioppa C, Assad JA 2006. Neurons in the orbitofrontal cortex encode economic value. Nature 441:223–26
    [Google Scholar]
  130. 130.  Barron HC, Dolan RJ, Behrens TEJ 2013. Online evaluation of novel choices by simultaneous representation of multiple memories. Nat. Neurosci. 16:1492–98
    [Google Scholar]
  131. 131.  Bogacz R, Brown E, Moehlis J, Holmes P, Cohen JD 2006. The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychol. Rev. 113:700
    [Google Scholar]
  132. 132.  Solway A, Botvinick MM 2012. Goal-directed decision making as probabilistic inference. Psychol. Rev. 119:120–54
    [Google Scholar]
  133. 133.  Resulaj A, Kiani R, Wolpert DM, Shadlen MN 2009. Changes of mind in decision-making. Nature 461:263–66
    [Google Scholar]
  134. 134.  Solway A, Botvinick MM 2015. Evidence integration in model-based tree search. PNAS 112:11708–13
    [Google Scholar]
  135. 135.  Daw ND, Niv Y, Dayan P 2005. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8:1704–11
    [Google Scholar]
  136. 136.  Wilson RC, Takahashi YK, Schoenbaum G, Niv Y 2014. Orbitofrontal cortex as a cognitive map of task space. Neuron 81:267–79
    [Google Scholar]
  137. 137.  Bornstein AM, Norman KA 2017. Reinstated episodic context guides sampling-based decisions for reward. Nat. Neurosci. 20:997–1003
    [Google Scholar]
  138. 138.  Jax S, Rosenbaum D 2007. Hand path priming in manual obstacle avoidance: evidence that the dorsal stream does not only control visually guided actions in real time. J. Exp. Psychol. Hum. Percept. Perform. 33:425–41
    [Google Scholar]
  139. 139.  Battaglia PW, Hamrick JB, Tenenbaum JB 2013. Simulation as an engine of physical scene understanding. PNAS 110:18327–32
    [Google Scholar]
  140. 140.  Srinivas A, Jabri A, Abbeel P, Levine S, Finn C 2018. Universal planning networks: learning generalizable representations for visuomotor control. Proceedings of the 35th International Conference on Machine Learning J Dy, A Krause4739–48 Proc. Mach. Learn. Res. Vol. 80. N.p.: PMLR
    [Google Scholar]
  141. 141.  Ghahramani Z 2015. Probabilistic machine learning and artificial intelligence. Nature 521:452–59
    [Google Scholar]
  142. 142.  Esposito M, Capelli P, Arber S 2014. Brainstem nucleus MdV mediates skilled forelimb motor tasks. Nature 508:351–56
    [Google Scholar]
  143. 143.  Azim E, Jiang J, Alstermark B, Jessell TM 2014. Skilled reaching relies on a V2a propriospinal internal copy circuit. Nature 508:357–63
    [Google Scholar]
/content/journals/10.1146/annurev-control-060117-105206
Loading
/content/journals/10.1146/annurev-control-060117-105206
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error