1932
0

Annual Review of Vision Science

Volume 10, 2024

Optimization in Visual Motion Estimation

  • Damon A. Clark1 and James E. Fitzgerald2,3
  • 1Department of Molecular, Cellular, and Developmental Biology, Yale University, New Haven, Connecticut, USA; email: [email protected] 2Department of Neurobiology, Northwestern University, Evanston, Illinois, USA; email: [email protected] 3Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, Virginia, USA
  • Vol. 10:23-46 (Volume publication date September 2024)
  • First published as a Review in Advance on April 25, 2024
  • Copyright © 2024 by the author(s).
    This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See credit lines of images or other third-party material in this article for license information.

Abstract

Sighted animals use visual signals to discern directional motion in their environment. Motion is not directly detected by visual neurons, and it must instead be computed from light signals that vary over space and time. This makes visual motion estimation a near universal neural computation, and decades of research have revealed much about the algorithms and mechanisms that generate directional signals. The idea that sensory systems are optimized for performance in natural environments has deeply impacted this research. In this article, we review the many ways that optimization has been used to quantitatively model visual motion estimation and reveal its underlying principles. We emphasize that no single optimization theory has dominated the literature. Instead, researchers have adeptly incorporated different computational demands and biological constraints that are pertinent to the specific brain system and animal model under study. The successes and failures of the resulting optimization models have thereby provided insights into how computational demands and biological constraints together shape neural computation.

Keyword(s): Bayes optimality, efficient coding, motion estimation, optimization, task optimization
Loading

Article metrics loading...

/content/journals/10.1146/annurev-vision-101623-025432
2024-09-15
2025-06-12
Loading full text...

Full text loading...

/deliver/fulltext/vision/10/1/annurev-vision-101623-025432.html?itemId=/content/journals/10.1146/annurev-vision-101623-025432&mimeType=html&fmt=ahah

Literature Cited

  1. Adelson EH, Bergen JR. 1985.. Spatiotemporal energy models for the perception of motion. . J. Opt. Soc. Am. A 2:(2):284–99
     [Crossref] [Google Scholar]
  2. Agrochao M, Tanaka R, Salazar-Gatzimas E, Clark DA. 2020.. Mechanism for analogous illusory motion perception in flies and humans. . PNAS 117:(37):23044–53
     [Crossref] [Google Scholar]
  3. Alexander E, Cai LT, Fuchs S, Hladnik TC, Zhang Y, et al. 2022.. Optic flow in the natural habitats of zebrafish supports spatial biases in visual self-motion estimation. . Curr. Biol. 32:(23):5008–21.e8
     [Crossref] [Google Scholar]
  4. Anderson PW. 1972.. More is different: broken symmetry and the nature of the hierarchical structure of science. . Science 177:(4047):393–96
     [Crossref] [Google Scholar]
  5. Anstis S. 1970.. Phi movement as a subtraction process. . Vis. Res. 10:(12):1411–30
     [Crossref] [Google Scholar]
  6. Atick JJ, Redlich AN. 1990.. Towards a theory of early visual processing. . Neural Comput. 2:(3):308–20
     [Crossref] [Google Scholar]
  7. Attneave F. 1954.. Some informational aspects of visual perception. . Psychol. Rev. 61:(3):183–93
     [Crossref] [Google Scholar]
  8. Baccus SA, Meister M. 2002.. Fast and slow contrast adaptation in retinal circuitry. . Neuron 36:(5):909–19
     [Crossref] [Google Scholar]
  9. Badwan BA, Creamer MS, Zavatone-Veth JA, Clark DA. 2019.. Dynamic nonlinearities enable direction opponency in Drosophila elementary motion detectors. . Nat. Neurosci. 22:(8):1318–26
     [Crossref] [Google Scholar]
  10. Bahroun Y, Chklovskii D, Sengupta A. 2019.. A similarity-preserving network trained on transformed images recapitulates salient features of the fly motion detection circuit. . In Advances in Neural Information Processing Systems 32, ed. H Wallach, H Larochelle, A Beygelzimer, F d'Alché-Buc, E Fox, R Garnett. San Diego: Neural Inf. Process. Syst. Found. https://proceedings.neurips.cc/paper_files/paper/2019/file/dab1263d1e6a88c9ba5e7e294def5e8b-Paper.pdf
    [Google Scholar]
  11. Barlow H, Levick WR. 1965.. The mechanism of directionally selective units in rabbit's retina. . J. Physiol. 178:(3):477–504
     [Crossref] [Google Scholar]
  12. Barlow HB. 1961.. Possible principles underlying the transformation of sensory messages. . In Sensory Communication, ed. W Rosenblith , pp. 217–34. Cambridge, MA:: MIT Press
     [Google Scholar]
  13. Bell AJ, Sejnowski TJ. 1997.. The “independent components” of natural scenes are edge filters. . Vis. Res. 37:(23):3327–38
     [Crossref] [Google Scholar]
  14. Bialek W, de Ruyter van Steveninck RR. 2005.. Features and dimensions: motion estimation in fly vision. . arXiv:q-bio/0505003 [q-bio.NC]
  15. Bigge R, Pfefferle M, Pfeiffer K, Stöckl A. 2021.. Natural image statistics in the dorsal and ventral visual field match a switch in flight behaviour of a hawkmoth. . Curr. Biol. 31:(6):R280–81
     [Crossref] [Google Scholar]
  16. Biswas T, Bishop WE, Fitzgerald JE. 2020.. Theoretical principles for illuminating sensorimotor processing with brain-wide neuronal recordings. . Curr. Opin. Neurobiol. 65::138–45
     [Crossref] [Google Scholar]
  17. Borst A. 2018.. A biophysical mechanism for preferred direction enhancement in fly motion vision. . PLOS Comput. Biol. 14:(6):e1006240
     [Crossref] [Google Scholar]
  18. Borst A, Egelhaaf M. 1989.. Principles of visual motion detection. . Trends Neurosci. 12:(8):297–306
     [Crossref] [Google Scholar]
  19. Borst A, Flanagin VL, Sompolinsky H. 2005.. Adaptation without parameter change: dynamic gain control in motion detection. . PNAS 102:(17):6172–76
     [Crossref] [Google Scholar]
  20. Brady N, Field DJ. 2000.. Local contrast in natural images: normalisation and coding efficiency. . Perception 29:(9):1041–55
     [Crossref] [Google Scholar]
  21. Brenner N, Bialek W, de Ruyter van Steveninck R. 2000.. Adaptive rescaling maximizes information transmission. . Neuron 26:(3):695–702
     [Crossref] [Google Scholar]
  22. Buckley CL, Kim CS, McGregor S, Seth AK. 2017.. The free energy principle for action and perception: a mathematical review. . J. Math. Psychol. 81::55–79
     [Crossref] [Google Scholar]
  23. Burge J, Geisler WS. 2015.. Optimal speed estimation in natural image movies predicts human performance. . Nat. Commun. 6::7900
     [Crossref] [Google Scholar]
  24. Carandini M, Heeger DJ. 2012.. Normalization as a canonical neural computation. . Nat. Rev. Neurosci. 13:(1):51–62
     [Crossref] [Google Scholar]
  25. Chen J, Mandel HB, Fitzgerald JE, Clark DA. 2019.. Asymmetric ON-OFF processing of visual motion cancels variability induced by the structure of natural scenes. . eLife 8::e47579
     [Crossref] [Google Scholar]
  26. Chiappe ME, Seelig JD, Reiser MB, Jayaraman V. 2010.. Walking modulates speed sensitivity in Drosophila motion vision. . Curr. Biol. 20:(16):1470–75
     [Crossref] [Google Scholar]
  27. Chin BM, Burge J. 2020.. Predicting the partition of behavioral variability in speed perception with naturalistic stimuli. . J. Neurosci. 40:(4):864–79
     [Crossref] [Google Scholar]
  28. Churchland PS, Sejnowski TJ. 1988.. Perspectives on cognitive neuroscience. . Science 242:(4879):741–45
     [Crossref] [Google Scholar]
  29. Clark DA, Demb JB. 2016.. Parallel computations in insect and mammalian visual motion processing. . Curr. Biol. 26:(20):R1062–72
     [Crossref] [Google Scholar]
  30. Clark DA, Fitzgerald JE, Ales JM, Gohl DM, Silies MA, et al. 2014.. Flies and humans share a motion estimation strategy that exploits natural scene statistics. . Nat. Neurosci. 17:(2):296–303
     [Crossref] [Google Scholar]
  31. Conway BR, Kitaoka A, Yazdanbakhsh A, Pack CC, Livingstone MS. 2005.. Neural basis for a powerful static motion illusion. . J. Neurosci. 25:(23):5651–56
     [Crossref] [Google Scholar]
  32. Creamer MS, Mano O, Clark DA. 2018.. Visual control of walking speed in Drosophila. . Neuron 100:(6):1460–73
     [Crossref] [Google Scholar]
  33. Dahmen HJ, Franz MO, Krapp HG. 2001.. Extracting egomotion from optic flow: limits of accuracy and neural matched filters. . In Motion Vision: Computational, Neural, and Ecological Constraints, ed. JM Zanker, J Zeil , pp. 143–68. Berlin:: Springer
     [Google Scholar]
  34. de Ruyter van Steveninck RR, Bialek W, Potters M, Carlson R. 1994.. Statistical adaptation and optimal estimation in movement computation by the blowfly visual system. . In Proceedings of IEEE International Conference on Systems, Man and Cybernetics, Vol. 1, pp. 302–7. Piscataway, NJ:: IEEE
     [Google Scholar]
  35. De Valois RL, Cottaris NP, Mahon LE, Elfar SD, Wilson JA. 2000.. Spatial and temporal receptive fields of geniculate and cortical cells and directional selectivity. . Vis. Res. 40:(27):3685–702
     [Crossref] [Google Scholar]
  36. Ding H, Smith RG, Poleg-Polsky A, Diamond JS, Briggman KL. 2016.. Species-specific wiring for direction selectivity in the mammalian retina. . Nature 535:(7610):105–10
     [Crossref] [Google Scholar]
  37. Drews MS, Leonhardt A, Pirogova N, Richter FG, Schuetzenberger A, et al. 2020.. Dynamic signal compression for robust motion vision in flies. . Curr. Biol. 30:(2):209–21
     [Crossref] [Google Scholar]
  38. Dror RO, O'Carroll DC, Laughlin SB. 2001.. Accuracy of velocity estimation by Reichardt correlators. . J. Opt. Soc. Am. A 18:(2):241–52
     [Crossref] [Google Scholar]
  39. Duffy CJ, Wurtz RH. 1991.. Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuli. . J. Neurophysiol. 65:(6):1329–45
     [Crossref] [Google Scholar]
  40. Ecke GA, Bruijns SA, Hölscher J, Mikulasch FA, Witschel T, et al. 2020.. Sparse coding predicts optic flow specificities of zebrafish pretectal neurons. . Neural Comput. Appl. 32:(11):6745–54
     [Crossref] [Google Scholar]
  41. Egelhaaf M, Borst A. 1989.. Transient and steady-state response properties of movement detectors. . J. Opt. Soc. Am. A 6:(1):116–27
     [Crossref] [Google Scholar]
  42. Fairhall AL, Lewen GD, Bialek W, de Ruyter van Steveninck RR. 2001.. Efficiency and ambiguity in an adaptive neural code. . Nature 412:(6849):787–92
     [Crossref] [Google Scholar]
  43. Farid H, Simoncelli EP. 2004.. Differentiation of discrete multidimensional signals. . IEEE Trans. Image Process. 13:(4):496–508
     [Crossref] [Google Scholar]
  44. Fennema CL, Thompson WB. 1979.. Velocity determination in scenes containing several moving objects. . Comput. Graph. Image Process. 9:(4):301–15
     [Crossref] [Google Scholar]
  45. Fitzgerald JE, Clark DA. 2015.. Nonlinear circuits for naturalistic visual motion estimation. . eLife 4::e09123
     [Crossref] [Google Scholar]
  46. Fitzgerald JE, Katsov AY, Clandinin TR, Schnitzer MJ. 2011.. Symmetries in stimulus statistics shape the form of visual motion estimators. . PNAS 108:(31):12909–14
     [Crossref] [Google Scholar]
  47. Fransen JW, Borghuis BG. 2017.. Temporally diverse excitation generates direction-selective responses in ON- and OFF-type retinal starburst amacrine cells. . Cell Rep. 18:(6):1356–65
     [Crossref] [Google Scholar]
  48. Fraser A, Wilcox KJ. 1979.. Perception of illusory movement. . Nature 281:(5732):565–66
     [Crossref] [Google Scholar]
  49. Girshick AR, Landy MS, Simoncelli EP. 2011.. Cardinal rules: Visual orientation perception reflects knowledge of environmental statistics. . Nat. Neurosci. 14:(7):926–32
     [Crossref] [Google Scholar]
  50. Gonzalez-Suarez AD, Zavatone-Veth JA, Chen J, Matulis CA, Badwan BA, Clark DA. 2022.. Excitatory and inhibitory neural dynamics jointly tune motion detection. . Curr. Biol. 32:(17):3659–75
     [Crossref] [Google Scholar]
  51. Gori S, Agrillo C, Dadda M, Bisazza A. 2014.. Do fish perceive illusory motion?. Sci. Rep. 4::6443
     [Crossref] [Google Scholar]
  52. Groschner LN, Malis JG, Zuidinga B, Borst A. 2022.. A biophysical account of multiplication by a single neuron. . Nature 603:(7899):119–23
     [Crossref] [Google Scholar]
  53. Gruntman E, Romani S, Reiser MB. 2018.. Simple integration of fast excitation and offset, delayed inhibition computes directional selectivity in Drosophila. . Nat. Neurosci. 21:(2):250–57
     [Crossref] [Google Scholar]
  54. Gruntman E, Romani S, Reiser MB. 2019.. The computation of directional selectivity in the Drosophila OFF motion pathway. . eLife 8::e50706
     [Crossref] [Google Scholar]
  55. Haag J, Denk W, Borst A. 2004.. Fly motion vision is based on Reichardt detectors regardless of the signal-to-noise ratio. . PNAS 101:(46):16333–38
     [Crossref] [Google Scholar]
  56. Hassenstein B, Reichardt W. 1956.. Systemtheoretische analyse der zeit-, reihenfolgen- und vorzeichenauswertung bei der bewegungsperzeption des rüsselkäfers chlorophanus. . Z. Naturforsch. B 11:(9–10):513–24
     [Crossref] [Google Scholar]
  57. Hausselt SE, Euler T, Detwiler PB, Denk W. 2007.. A dendrite-autonomous mechanism for direction selectivity in retinal starburst amacrine cells. . PLOS Biol. 5:(7):e185
     [Crossref] [Google Scholar]
  58. Heeger DJ. 1991.. Nonlinear model of neural responses in cat visual cortex. . In Computational Models of Visual Processing, ed. MS Landy, JA Movshon , pp. 119–33. Cambridge, MA:: MIT Press
     [Google Scholar]
  59. Heeger DJ, Jepson AD. 1992.. Subspace methods for recovering rigid motion I: algorithm and implementation. . Int. J. Comput. Vis. 7:(2):95–117
     [Crossref] [Google Scholar]
  60. Henning M, Ramos-Traslosheros G, Gür B, Silies M. 2022.. Populations of local direction–selective cells encode global motion patterns generated by self-motion. . Sci. Adv. 8:(31):eabi7112
     [Crossref] [Google Scholar]
  61. Holman JG, Lai WW, Pichler P, Saska D, Lagnado L, Buckley CL. 2023.. A behavioral and modeling study of control algorithms underlying the translational optomotor response in larval zebrafish with implications for neural circuit function. . PLOS Comput. Biol. 19:(2):e1010924
     [Crossref] [Google Scholar]
  62. Hu Q, Victor JD. 2010.. A set of high-order spatiotemporal stimuli that elicit motion and reverse-phi percepts. . J. Vis. 10:(3):9
     [Crossref] [Google Scholar]
  63. Jagadeesh B, Wheat HS, Ferster D. 1993.. Linearity of summation of synaptic potentials underlying direction selectivity in simple cells of the cat visual cortex. . Science 262:(5141):1901–4
     [Crossref] [Google Scholar]
  64. Kim JS, Greene MJ, Zlateski A, Lee K, Richardson M, et al. 2014.. Space–time wiring specificity supports direction selectivity in the retina. . Nature 509:(7500):331–36
     [Crossref] [Google Scholar]
  65. Kitaoka A, Ashida H. 2003.. Phenomenal characteristics of the peripheral drift illusion. . Vision 15:(4):261–62
     [Google Scholar]
  66. Koch K, McLean J, Segev R, Freed MA, Berry MJ, et al. 2006.. How much the eye tells the brain. . Curr. Biol. 16:(14):1428–34
     [Crossref] [Google Scholar]
  67. Koenderink JJ, van Doorn AJ. 1987.. Facts on optic flow. . Biol. Cybernet. 56:(4):247–54
     [Crossref] [Google Scholar]
  68. Krapp HG, Hengstenberg R. 1996.. Estimation of self-motion by optic flow processing in single visual interneurons. . Nature 384:(6608):463–66
     [Crossref] [Google Scholar]
  69. Kretschmer F, Tariq M, Chatila W, Wu B, Badea TC. 2017.. Comparison of optomotor and optokinetic reflexes in mice. . J. Neurophysiol. 118:(1):300–16
     [Crossref] [Google Scholar]
  70. Lappalainen JK, Tschopp FD, Prakhya S, McGill M, Nern A, et al. 2023.. Connectome-constrained deep mechanistic networks predict neural responses across the fly visual system at single-neuron resolution. . bioRxiv 2023.03.11.532232. https://doi.org/10.1101/2023.03.11.532232
  71. Laughlin S. 1981.. A simple coding procedure enhances a neuron's information capacity. . Z. Naturforsch. C 36:(9–10):910–12
     [Crossref] [Google Scholar]
  72. Leong JCS, Esch JJ, Poole B, Ganguli S, Clandinin TR. 2016.. Direction selectivity in Drosophila emerges from preferred-direction enhancement and null-direction suppression. . J. Neurosci. 36:(31):8078–92
     [Crossref] [Google Scholar]
  73. Leonhardt A, Ammer G, Meier M, Serbe E, Bahl A, Borst A. 2016.. Asymmetry of Drosophila ON and OFF motion detectors enhances real-world velocity estimation. . Nat. Neurosci. 19:(5):706–15
     [Crossref] [Google Scholar]
  74. Li Z. 1996.. A theory of the visual motion coding in the primary visual cortex. . Neural Comput. 8:(4):705–30
     [Crossref] [Google Scholar]
  75. Liu B, Hong A, Rieke F, Manookin MB. 2021.. Predictive encoding of motion begins in the primate retina. . Nat. Neurosci. 24:(9):1280–91
     [Crossref] [Google Scholar]
  76. Livingstone MS, Conway BR. 2003.. Substructure of direction-selective receptive fields in macaque V1. . J. Neurophysiol. 89:(5):2743–59
     [Crossref] [Google Scholar]
  77. Maisak MS, Haag J, Ammer G, Serbe E, Meier M, et al. 2013.. A directional tuning map of Drosophila elementary motion detectors. . Nature 500:(7461):212–16
     [Crossref] [Google Scholar]
  78. Mano O, Creamer MS, Badwan BA, Clark DA. 2021.. Predicting individual neuron responses with anatomically constrained task optimization. . Curr. Biol. 31:(18):4062–75
     [Crossref] [Google Scholar]
  79. Manookin MB, Patterson SS, Linehan CM. 2018.. Neural mechanisms mediating motion sensitivity in parasol ganglion cells of the primate retina. . Neuron 97:(6):1327–40
     [Crossref] [Google Scholar]
  80. Markov DA, Petrucco L, Kist AM, Portugues R. 2021.. A cerebellar internal model calibrates a feedback controller involved in sensorimotor control. . Nat. Commun. 12::6694
     [Crossref] [Google Scholar]
  81. Marr D, Poggio T. 1976.. From understanding computation to understanding neural circuitry. Tech. Rep. , Mass. Inst. Technol., Cambridge:
     [Google Scholar]
  82. Matulis CA, Chen J, Gonzalez-Suarez AD, Behnia R, Clark DA. 2020.. Heterogeneous temporal contrast adaptation in Drosophila direction-selective circuits. . Curr. Biol. 30:(2):222–36
     [Crossref] [Google Scholar]
  83. Mauss AS, Pankova K, Arenz A, Nern A, Rubin GM, Borst A. 2015.. Neural circuit to integrate opposing motions in the visual field. . Cell 162:(2):351–62
     [Crossref] [Google Scholar]
  84. McLean J, Palmer LA. 1989.. Contribution of linear spatiotemporal receptive field structure to velocity selectivity of simple cells in area 17 of cat. . Vis. Res. 29:(6):675–79
     [Crossref] [Google Scholar]
  85. Młynarski W, Hledík M, Sokolowski TR, Tkačik G. 2021.. Statistical analysis and optimality of neural systems. . Neuron 109:(7):1227–41
     [Crossref] [Google Scholar]
  86. Mo CH, Koch C. 2003.. Modeling reverse-phi motion-selective neurons in cortex: double synaptic-veto mechanism. . Neural Comput. 15:(4):735–59
     [Crossref] [Google Scholar]
  87. Murray KT, Wang MB, Lynch N. 2022.. Emergence of direction-selective retinal cell types in task-optimized deep learning models. . J. Comput. Biol. 29:(4):370–81
     [Crossref] [Google Scholar]
  88. Naumann EA, Fitzgerald JE, Dunn TW, Rihel J, Sompolinsky H, Engert F. 2016.. From whole-brain data to functional circuit models: the zebrafish optomotor response. . Cell 167:(4):947–60
     [Crossref] [Google Scholar]
  89. Nitzany EI, Loe ME, Palmer SE, Victor JD. 2016.. Perceptual interaction of local motion signals. . J. Vis. 16:(14):22
     [Crossref] [Google Scholar]
  90. Nitzany EI, Menda G, Shamble PS, Golden JR, Hu Q, et al. 2017.. Neural computations combine low- and high-order motion cues similarly, in dragonfly and monkey. . bioRxiv 240101. https://doi.org/10.1101/240101
  91. Nitzany EI, Victor JD. 2014.. The statistics of local motion signals in naturalistic movies. . J. Vis. 14:(4):10
     [Crossref] [Google Scholar]
  92. Orger MB, Smear MC, Anstis SM, Baier H. 2000.. Perception of Fourier and non-Fourier motion by larval zebrafish. . Nat. Neurosci. 3:(11):1128–33
     [Crossref] [Google Scholar]
  93. Pachitariu M, Sahani M. 2012.. Learning visual motion in recurrent neural networks. . Adv. Neural Inform. Proc. Syst. 25::1322–30
     [Google Scholar]
  94. Palmer SE, Marre O, Berry MJ, Bialek W. 2015.. Predictive information in a sensory population. . PNAS 112:(22):6908–13
     [Crossref] [Google Scholar]
  95. Perrone JA, Stone LS. 1994.. A model of self-motion estimation within primate extrastriate visual cortex. . Vis. Res. 34:(21):2917–38
     [Crossref] [Google Scholar]
  96. Poggio T, Reichardt W. 1973.. Considerations on models of movement detection. . Kybernetik 13:(4):223–27
     [Crossref] [Google Scholar]
  97. Poleg-Polsky A, Ding H, Diamond JS. 2018.. Functional compartmentalization within starburst amacrine cell dendrites in the retina. . Cell Rep. 22:(11):2898–908
     [Crossref] [Google Scholar]
  98. Portugues R, Feierstein CE, Engert F, Orger MB. 2014.. Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior. . Neuron 81:(6):1328–43
     [Crossref] [Google Scholar]
  99. Potters M, Bialek W. 1994.. Statistical mechanics and visual signal processing. . J. Phys. I 4:(11):1755–75
     [Google Scholar]
  100. Rao R, Ruderman D. 1998.. Learning Lie groups for invariant visual perception. . Adv. Neural Inform. Proc. Syst. 11::810–16
     [Google Scholar]
  101. Rao R, Sejnowski TJ. 1999.. Predictive sequence learning in recurrent neocortical circuits. . Adv. Neural Inform. Proc. Syst. 12::164–70
     [Google Scholar]
  102. Reisenman C, Haag J, Borst A. 2003.. Adaptation of response transients in fly motion vision. I. Experiments. . Vis. Res. 43:(11):1293–309
     [Crossref] [Google Scholar]
  103. Richards BA, Lillicrap TP, Beaudoin P, Bengio Y, Bogacz R, et al. 2019.. A deep learning framework for neuroscience. . Nat. Neurosci. 22:(11):1761–70
     [Crossref] [Google Scholar]
  104. Rideaux R, Welchman AE. 2020.. But still it moves: Static image statistics underlie how we see motion. . J. Neurosci. 40:(12):2538–52
     [Crossref] [Google Scholar]
  105. Rieke F. 2001.. Temporal contrast adaptation in salamander bipolar cells. . J. Neurosci. 21:(23):9445–54
     [Crossref] [Google Scholar]
  106. Ruderman D, Bialek W. 1993.. Statistics of natural images: scaling in the woods. . Adv. Neural Inform. Proc. Syst. 6::551–58
     [Google Scholar]
  107. Rust NC, Mante V, Simoncelli EP, Movshon JA. 2006.. How MT cells analyze the motion of visual patterns. . Nat. Neurosci. 9:(11):1421–31
     [Crossref] [Google Scholar]
  108. Rust NC, Schwartz O, Movshon JA, Simoncelli EP. 2005.. Spatiotemporal elements of macaque V1 receptive fields. . Neuron 46:(6):945–56
     [Crossref] [Google Scholar]
  109. Sabbah S, Gemmer JA, Bhatia-Lin A, Manoff G, Castro G, et al. 2017.. A retinal code for motion along the gravitational and body axes. . Nature 546:(7659):492–97
     [Crossref] [Google Scholar]
  110. Sachdeva V, Mora T, Walczak AM, Palmer SE. 2021.. Optimal prediction with resource constraints using the information bottleneck. . PLOS Comput. Biol. 17:(3):e1008743
     [Crossref] [Google Scholar]
  111. Safran MN, Flanagin VL, Borst A, Sompolinsky H. 2007.. Adaptation and information transmission in fly motion detection. . J. Neurophysiol. 98:(6):3309–20
     [Crossref] [Google Scholar]
  112. Salazar-Gatzimas E, Agrochao M, Fitzgerald JE, Clark DA. 2018.. The neuronal basis of an illusory motion percept is explained by decorrelation of parallel motion pathways. . Curr. Biol. 28:(23):3748–62
     [Crossref] [Google Scholar]
  113. Salazar-Gatzimas E, Chen J, Creamer MS, Mano O, Mandel HB, et al. 2016.. Direct measurement of correlation responses in Drosophila elementary motion detectors reveals fast timescale tuning. . Neuron 92:(1):227–39
     [Crossref] [Google Scholar]
  114. Saleem AB, Ayaz A, Jeffery KJ, Harris KD, Carandini M. 2013.. Integration of visual motion and locomotion in mouse visual cortex. . Nat. Neurosci. 16:(12):1864–69
     [Crossref] [Google Scholar]
  115. Salisbury JM, Palmer SE. 2016.. Optimal prediction in the retina and natural motion statistics. . J. Stat. Phys. 162:(5):1309–23
     [Crossref] [Google Scholar]
  116. Schachter MJ, Oesch N, Smith RG, Taylor WR. 2010.. Dendritic spikes amplify the synaptic signal to enhance detection of motion in a simulation of the direction-selective ganglion cell. . PLOS Comput. Biol. 6:(8):e1000899
     [Crossref] [Google Scholar]
  117. Schwartz O, Pillow JW, Rust NC, Simoncelli EP. 2006.. Spike-triggered neural characterization. . J. Vis. 6:(4):13
     [Crossref] [Google Scholar]
  118. Schwartz O, Simoncelli EP. 2001.. Natural signal statistics and sensory gain control. . Nat. Neurosci. 4:(8):819–25
     [Crossref] [Google Scholar]
  119. Simoncelli EP. 1993.. Distributed representation and analysis of visual motion. PhD Thesis , Mass. Inst. Technol., Cambridge:
     [Google Scholar]
  120. Simoncelli EP, Adelson EH, Heeger DJ. 1991.. Probability distributions of optical flow. . In Proceedings of the 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 310–15. Piscataway, NJ:: IEEE
     [Google Scholar]
  121. Simoncelli EP, Heeger DJ. 1998.. A model of neuronal responses in visual area MT. . Vis. Res. 38:(5):743–61
     [Crossref] [Google Scholar]
  122. Simoncelli EP, Olshausen BA. 2001.. Natural image statistics and neural representation. . Annu. Rev. Neurosci. 24::1193–216
     [Crossref] [Google Scholar]
  123. Sinha SR, Bialek W, de Ruyter van Steveninck RR. 2021.. Optimal local estimates of visual motion in a natural environment. . Phys. Rev. Lett. 126:(1):018101
     [Crossref] [Google Scholar]
  124. Srinivasan MV, Laughlin SB, Dubs A. 1982.. Predictive coding: a fresh view of inhibition in the retina. . Proc. R. Soc. Lond. B 216:(1205):427–59
     [Crossref] [Google Scholar]
  125. Srivastava P, de Rosenroll G, Matsumoto A, Michaels T, Turple Z, et al. 2022.. Spatiotemporal properties of glutamate input support direction selectivity in the dendrites of retinal starburst amacrine cells. . eLife 11::e81533
     [Crossref] [Google Scholar]
  126. Sterling P, Laughlin S. 2015.. Principles of Neural Design. Cambridge, MA:: MIT Press
     [Google Scholar]
  127. Still S. 2009.. Information-theoretic approach to interactive learning. . Europhys. Lett. 85:(2):28005
     [Crossref] [Google Scholar]
  128. Stocker AA, Simoncelli EP. 2006.. Noise characteristics and prior expectations in human visual speed perception. . Nat. Neurosci. 9:(4):578–85
     [Crossref] [Google Scholar]
  129. Stöckl A, O'Carroll D, Warrant E. 2017.. Higher-order neural processing tunes motion neurons to visual ecology in three species of hawkmoths. . Proc. R. Soc. B 284:(1857):20170880
     [Crossref] [Google Scholar]
  130. Suarez H, Koch C, Douglas R. 1995.. Modeling direction selectivity of simple cells in striate visual cortex within the framework of the canonical microcircuit. . J. Neurosci. 15:(10):6700–19
     [Crossref] [Google Scholar]
  131. Taub E, Victor JD, Conte MM. 1997.. Nonlinear preprocessing in short-range motion. . Vis. Res. 37:(11):1459–77
     [Crossref] [Google Scholar]
  132. Tishby N, Pereira FC, Bialek W. 2000.. The information bottleneck method. . arXiv:physics/0004057 [physics.data-an]
  133. Tishby N, Polani D. 2010.. Information theory of decisions and actions. . In Perception-Action Cycle: Models, Architectures, and Hardware, ed. V Cutsuridis, A Hussain, JG Taylor , pp. 601–36. Berlin:: Springer
     [Google Scholar]
  134. Torre V, Poggio T. 1978.. A synaptic mechanism possibly underlying directional selectivity to motion. . Proc. R. Soc. Lond. B 202:(1148):409–16
     [Crossref] [Google Scholar]
  135. Turner MH, Sanchez Giraldo LG, Schwartz O, Rieke F. 2019.. Stimulus- and goal-oriented frameworks for understanding natural vision. . Nat. Neurosci. 22:(1):15–24
     [Crossref] [Google Scholar]
  136. Van Hateren JH, van der Schaaf A. 1998.. Independent component filters of natural images compared with simple cells in primary visual cortex. . Proc. R. Soc. Lond. B 265:(1394):359–66
     [Crossref] [Google Scholar]
  137. Van Santen JP, Sperling G. 1985.. Elaborated Reichardt detectors. . J. Opt. Soc. Am. A 2:(2):300–21
     [Crossref] [Google Scholar]
  138. von Helmholtz H. 2013 (1924).. Treatise on Physiological Optics, Volume III. North Chelmsford, MA:: Courier Corp.
     [Google Scholar]
  139. Wang S, Borst A, Zaslavsky N, Tishby N, Segev I. 2017.. Efficient encoding of motion is mediated by gap junctions in the fly visual system. . PLOS Comput. Biol. 13:(12):e1005846
     [Crossref] [Google Scholar]
  140. Wang S, Segev I, Borst A, Palmer S. 2021.. Maximally efficient prediction in the early fly visual system may support evasive flight maneuvers. . PLOS Comput. Biol. 17:(5):e1008965
     [Crossref] [Google Scholar]
  141. Watanabe E, Kitaoka A, Sakamoto K, Yasugi M, Tanaka K. 2018.. Illusory motion reproduced by deep neural networks trained for prediction. . Front. Psychol. 9::345
     [Crossref] [Google Scholar]
  142. Wei W. 2018.. Neural mechanisms of motion processing in the mammalian retina. . Annu. Rev. Vis. Sci. 4::165–92
     [Crossref] [Google Scholar]
  143. Weiss Y, Simoncelli EP, Adelson EH. 2002.. Motion illusions as optimal percepts. . Nat. Neurosci. 5:(6):598–604
     [Crossref] [Google Scholar]
  144. Wienecke CF, Leong JC, Clandinin TR. 2018.. Linear summation underlies direction selectivity in Drosophila. . Neuron 99:(4):680–88
     [Crossref] [Google Scholar]
  145. Yamins DL, DiCarlo JJ. 2016.. Using goal-driven deep learning models to understand sensory cortex. . Nat. Neurosci. 19:(3):356–65
     [Crossref] [Google Scholar]
  146. Yang E, Zwart MF, James B, Rubinov M, Wei Z, et al. 2022.. A brainstem integrator for self-location memory and positional homeostasis in zebrafish. . Cell 185:(26):5011–27
     [Crossref] [Google Scholar]
  147. Yang HH, Clandinin TR. 2018.. Elementary motion detection in Drosophila: algorithms and mechanisms. . Annu. Rev. Vis. Sci. 4::143–63
     [Crossref] [Google Scholar]
  148. Yildizoglu T, Riegler C, Fitzgerald JE, Portugues R. 2020.. A neural representation of naturalistic motion-guided behavior in the zebrafish brain. . Curr. Biol. 30:(12):2321–33
     [Crossref] [Google Scholar]
  149. Zavatone-Veth JA, Badwan BA, Clark DA. 2020.. A minimal synaptic model for direction selective neurons in Drosophila. . J. Vis. 20:(2):2
     [Crossref] [Google Scholar]
  150. Zhang Y, Huang R, Nörenberg W, Arrenberg AB. 2022.. A robust receptive field code for optic flow detection and decomposition during self-motion. . Curr. Biol. 32:(11):2505–16
     [Crossref] [Google Scholar]
/content/journals/10.1146/annurev-vision-101623-025432
Loading
Optimization in Visual Motion Estimation
Annual Review of Vision Science 10, 23 (2024); https://doi.org/10.1146/annurev-vision-101623-025432
/content/journals/10.1146/annurev-vision-101623-025432
/content/journals/10.1146/annurev-vision-101623-025432
Loading

Data & Media loading...

Supplemental Materials

  • Supplemental Video 1

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error