The visual system must recover important properties of the external environment if its host is to survive. Because the retinae are effectively two-dimensional but the world is three-dimensional (3D), the patterns of stimulation both within and across the eyes must be used to infer the distal stimulus—the environment—in all three dimensions. Moreover, animals and elements in the environment move, which means the input contains rich temporal information. Here, in addition to reviewing the literature, we discuss how and why prior work has focused on purported isolated systems (e.g., stereopsis) or cues (e.g., horizontal disparity) that do not necessarily map elegantly on to the computations and complex patterns of stimulation that arise when visual systems operate within the real world. We thus also introduce the binoptic flow field (BFF) as a description of the 3D motion information available in realistic environments, which can foster the use of ecologically valid yet well-controlled stimuli. Further, it can help clarify how future studies can more directly focus on the computations and stimulus properties the visual system might use to support perception and behavior in a dynamic 3D world.


Article metrics loading...

Loading full text...

Full text loading...


Literature Cited

  1. Adelson EH, Bergen JR. 1985. Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A 2:2284–99 [Google Scholar]
  2. Adelson EH, Bergen JR. 1991. The plenoptic function and elements of early vision. Computational Models of Visual Processing MS Landy, JA Movshon 3–20 Cambridge, MA: MIT Press [Google Scholar]
  3. Adelson EH, Movshon JA. 1982. Phenomenal coherence of moving visual patterns. Nature 300:523–25 [Google Scholar]
  4. Albright TD. 1984. Direction and orientation selectivity of neurons in visual area MT of the macaque. J. Neurophysiol. 52:61106–30 [Google Scholar]
  5. Allen B, Haun AM, Hanley T, Green CS, Rokers B. 2015. Optimal combination of the binocular cues to 3D motion. Invest. Ophthalmol. Vis. Sci. 56:127589–96 http://doi.org/10.1167/iovs.15-17696 [Crossref] [Google Scholar]
  6. Anstis SM. 1970. Phi movement as a subtraction process. Vis. Res. 10:1411–30 [Google Scholar]
  7. Backus BT, Banks MS, van Ee R, Crowell JA. 1999. Horizontal and vertical disparity, eye position, and stereoscopic slant perception. Vis. Res. 39:61143–70 [Google Scholar]
  8. Baker PM, Bair W. 2016. A model of binocular motion integration in MT neurons. J. Neurosci. 36:246563–82 http://doi.org/10.1523/JNEUROSCI.3213-15.2016 [Crossref] [Google Scholar]
  9. Barendregt M, Dumoulin SO, Rokers B. 2014. Stereomotion scotomas occur after binocular combination. Vis. Res. 105:92–99 http://doi.org/10.1016/j.visres.2014.09.008 [Crossref] [Google Scholar]
  10. Barendregt M, Dumoulin SO, Rokers B. 2016. Impaired velocity processing reveals an agnosia for motion in depth. Psychol. Sci. 27:111474–85 http://doi.org/10.1177/0956797616663488 [Crossref] [Google Scholar]
  11. Barendregt M, Harvey BM, Rokers B, Dumoulin SO. 2015. Transformation from a retinal to a cyclopean representation in human visual cortex. Curr. Biol. 25:151982–87 http://doi.org/10.1016/j.cub.2015.06.003 [Crossref] [Google Scholar]
  12. Blakemore C. 1970. The range and scope of binocular depth discrimination in man. J. Physiol. 211:3599–622 [Google Scholar]
  13. Blakemore C, Campbell FW. 1969. On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images. J. Physiol. 203:1237–60 [Google Scholar]
  14. Bonnen K, Czuba T, Kohn A, Cormack L, Huk A. 2017. Encoding and decoding in populations with non-Gaussian tuning: the example of 3D motion Presented at Cosyne 2017, Feb. 23–28 Salt Lake City, UT: [Google Scholar]
  15. Born RT, Bradley DC. 2005. Structure and function of visual area MT. Annu. Rev. Neurosci. 28:157–89 http://doi.org/10.1146/annurev.neuro.26.041002.131052 [Crossref] [Google Scholar]
  16. Braddick O. 1974. A short range process in apparent motion. Vis. Res. 14:519–27 [Google Scholar]
  17. Bremmer F, Duhamel J-R, Ben Hamed S, Graf W. 2002. Heading encoding in the macaque ventral intraparietal area (VIP). Eur. J. Neurosci. 16:81554–68 [Google Scholar]
  18. Bremmer F, Kubischik M, Pekel M, Lappe M, Hoffmann KP. 1999. Linear vestibular self-motion signals in monkey medial superior temporal area. Ann. N. Y. Acad. Sci. 871:272–81 [Google Scholar]
  19. Britten KH. 2008. Mechanisms of self-motion perception. Annu. Rev. Neurosci. 31:389–410 http://doi.org/10.1146/annurev.neuro.29.051605.112953 [Crossref] [Google Scholar]
  20. Britten KH, Shadlen MN, Newsome WT, Movshon JA. 1992. The analysis of visual motion: a comparison of neuronal and psychophysical performance. J. Neurosci. 12:124745–65 [Google Scholar]
  21. Britten KH, Shadlen MN, Newsome WT, Movshon JA. 1993. Responses of neurons in macaque MT to stochastic motion signals. Vis. Neurosci. 10:61157–69 http://doi.org/10.1017/S0952523800010269 [Crossref] [Google Scholar]
  22. Britten KH, van Wezel RJ. 1998. Electrical microstimulation of cortical area MST biases heading perception in monkeys. Nat. Neurosci. 1:159–63 http://doi.org/10.1038/259 [Crossref] [Google Scholar]
  23. Britten KH, van Wezel RJA. 2002. Area MST and heading perception in macaque monkeys. Cereb. Cortex 12:7692–701 [Google Scholar]
  24. Brooks KR. 2002a. Interocular velocity difference contributes to stereomotion speed perception. J. Vis. 2:32 http://doi.org/10:1167/2.3.2 [Crossref] [Google Scholar]
  25. Brooks KR. 2002b. Monocular motion adaptation affects the perceived trajectory of stereomotion. J. Exp. Psychol. Hum. Percept. Perform. 28:61470–82 [Google Scholar]
  26. Brooks KR, Stone LS. 2004. Stereomotion speed perception: contributions from both changing disparity and interocular velocity difference over a range of relative disparities. J. Vis. 4:126 http://doi.org/10.1167/4.12.6 [Crossref] [Google Scholar]
  27. Brooks KR, Stone LS. 2006. Stereomotion suppression and the perception of speed: accuracy and precision as a function of 3D trajectory. J. Vis. 6:116 http://doi.org/10.1167/6.11.6 [Crossref] [Google Scholar]
  28. Carney T, Shadlen MN. 1993. Dichoptic activation of the early motion system. Vis. Res. 33:141977–95 [Google Scholar]
  29. Chen A, Gu Y, Liu S, DeAngelis GC, Angelaki DE. 2016. Evidence for a causal contribution of macaque vestibular, but not intraparietal, cortex to heading perception. J. Neurosci. 36:133789–98 http://doi.org/10.1523/JNEUROSCI.2485-15.2016 [Crossref] [Google Scholar]
  30. Chen X, DeAngelis GC, Angelaki DE. 2014. Eye-centered visual receptive fields in the ventral intraparietal area. J. Neurophysiol. 112:2353–61 http://doi.org/10.1152/jn.00057.2014 [Crossref] [Google Scholar]
  31. Cumming BG. 1995. The relationship between stereoacuity and stereomotion thresholds. Perception 24:1105–14 http://doi.org/10.1068/p240105 [Crossref] [Google Scholar]
  32. Cumming BG, Parker AJ. 1994. Binocular mechanisms for detecting motion-in-depth. Vis. Res. 34:4483–95 [Google Scholar]
  33. Cynader M, Regan D. 1978. Neurones in cat parastriate cortex sensitive to the direction of motion in three-dimensional space. J. Physiol. 274:549–69 [Google Scholar]
  34. Cynader M, Regan D. 1982. Neurons in cat visual cortex tuned to the direction of motion in depth: effect of positional disparity. Vis. Res. 22:8967–82 [Google Scholar]
  35. Czuba TB, Huk AC, Cormack LK, Kohn A. 2014. Area MT encodes three-dimensional motion. J. Neurosci. 34:4715522–33 http://doi.org/10.1523/JNEUROSCI.1081-14.2014 [Crossref] [Google Scholar]
  36. Czuba TB, Rokers B, Guillet K, Huk AC, Cormack LK. 2011. Three-dimensional motion aftereffects reveal distinct direction-selective mechanisms for binocular processing of motion through depth. J. Vis. 11:1018 http://doi.org/10.1167/11.10.18 [Crossref] [Google Scholar]
  37. Czuba TB, Rokers B, Huk AC, Cormack LK. 2010. Speed and eccentricity tuning reveal a central role for the velocity-based cue to 3D visual motion. J. Neurophysiol. 104:52886–99 http://doi.org/10.1152/jn.00585.2009 [Crossref] [Google Scholar]
  38. Czuba TB, Rokers B, Huk AC, Cormack LK. 2012. To CD or not to CD: Is there a 3D motion aftereffect based on changing disparities?. J. Vis. 12:47 http://doi.org/10.1167/12.4.7 [Crossref] [Google Scholar]
  39. DeAngelis GC, Cumming BG, Newsome WT. 1998. Cortical area MT and the perception of stereoscopic depth. Nature 394:6694677–80 http://doi.org/10.1038/29299 [Crossref] [Google Scholar]
  40. DeAngelis GC, Uka T. 2003. Coding of horizontal disparity and velocity by MT neurons in the alert macaque. J. Neurophysiol. 89:21094–111 http://doi.org/10.1152/jn.00717.2002 [Crossref] [Google Scholar]
  41. Duffy CJ. 1998. MST neurons respond to optic flow and translational movement. J. Neurophysiol. 80:41816–27 [Google Scholar]
  42. Duffy CJ, Wurtz RH. 1991. Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuli. J. Neurophysiol. 65:61329–45 [Google Scholar]
  43. Duhamel JR, Bremmer F, Ben Hamed S, Graf W. 1997. Spatial invariance of visual receptive fields in parietal cortex neurons. Nature 389:6653845–48 http://doi.org/10.1038/39865 [Crossref] [Google Scholar]
  44. Duhamel JR, Colby CL, Goldberg ME. 1998. Ventral intraparietal area of the macaque: congruent visual and somatic response properties. J. Neurophysiol. 79:1126–36 [Google Scholar]
  45. Fernandez JM, Farell B. 2005. Seeing motion in depth using inter-ocular velocity differences. Vis. Res. 45:212786–98 http://doi.org/10.1016/j.visres.2005.05.021 [Crossref] [Google Scholar]
  46. Fernandez JM, Farell B. 2006. Motion in depth from interocular velocity differences revealed by differential motion aftereffect. Vis. Res. 46:8–91307–17 http://doi.org/10.1016/j.visres.2005.10.025 [Crossref] [Google Scholar]
  47. Gardner JL, Merriam EP, Movshon JA, Heeger DJ. 2008. Maps of visual space in human occipital cortex are retinotopic, not spatiotopic. J. Neurosci. 28:153988–99 http://doi.org/10.1523/JNEUROSCI.5476-07.2008 [Crossref] [Google Scholar]
  48. Gibson JJ. 1950. The Perception of the Visual World Boston, MA: Houghton Mifflin [Google Scholar]
  49. Gibson JJ. 1979. The Ecological Approach to Visual Perception New York: Psychol. Press [Google Scholar]
  50. Greer D, Joo SJ, Cormack L, Huk A. 2016. Global eye-specific motion signal for three-dimensional motion processing revealed through adaptation. J. Vis. 16:12182 http://doi.org/10.1167/16.12.182 [Crossref] [Google Scholar]
  51. Gu Y, DeAngelis GC, Angelaki DE. 2012. Causal links between dorsal medial superior temporal area neurons and multisensory heading perception. J. Neurosci. 32:72299–313 http://doi.org/10.1523/JNEUROSCI.5154-11.2012 [Crossref] [Google Scholar]
  52. Harris JM, Rushton SK. 2003. Poor visibility of motion in depth is due to early motion averaging. Vis. Res. 43:4385–92 [Google Scholar]
  53. Hong X, Regan D. 1989. Visual field defects for unidirectional and oscillatory motion in depth. Vis. Res. 29:7809–19 [Google Scholar]
  54. Joo SJ, Czuba TB, Cormack LK, Huk AC. 2016. Separate perceptual and neural processing of velocity- and disparity-based 3D motion signals. J. Neurosci. 36:4210791–802 http://doi.org/10.1523/JNEUROSCI.1298-16.2016 [Crossref] [Google Scholar]
  55. Julesz B. 1971. Foundations of Cyclopean Perception Chicago: Univ. Chicago Press [Google Scholar]
  56. Knapen T, Rolfs M, Cavanagh P. 2009. The reference frame of the motion aftereffect is retinotopic. J. Vis. 9:516 http://doi.org/10.1167/9.5.16 [Crossref] [Google Scholar]
  57. Lagae L, Maes H, Raiguel S, Xiao DK, Orban GA. 1994. Responses of macaque STS neurons to optic flow components: a comparison of areas MT and MST. J. Neurophysiol. 71:51597–626 [Google Scholar]
  58. Lages M, Heron S. 2010. On the inverse problem of binocular 3D motion perception. PLOS Comput. Biol. 6:11e1000999 http://doi.org/10.1371/journal.pcbi.1000999 [Crossref] [Google Scholar]
  59. Land MF, McLeod P. 2000. From eye movements to actions: how batsmen hit the ball. Nat. Neurosci. 3:121340–45 http://doi.org/10.1038/81887 [Crossref] [Google Scholar]
  60. Land MF, Nilsson DE. 2012. Animal Eyes Oxford, UK: Oxford Univ. Press, 2nd ed.. [Google Scholar]
  61. Lappe M. 2000. Computational mechanisms for optic flow analysis in primate cortex. Int. Rev. Neurobiol. 44:235–68 [Google Scholar]
  62. Likova LT, Tyler CW. 2007. Stereomotion processing in the human occipital cortex. NeuroImage 38:2293–305 http://doi.org/10.1016/j.neuroimage.2007.06.039 [Crossref] [Google Scholar]
  63. Maunsell JHR, Newsome WT. 1987. Visual processing in monkey extrastriate cortex. Annu. Rev. Neurosci. 10:363–401 http://doi.org/10.1146/annurev.ne.10.030187.002051 [Crossref] [Google Scholar]
  64. Maunsell JHR, Van Essen DC. 1983. Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. J. Neurophysiol. 49:51148–67 [Google Scholar]
  65. Mitchell DE, Reardon J, Muir DW. 1975. Interocular transfer of the motion after-effect in normal and stereoblind observers. Exp. Brain Res. 22:2163–73 [Google Scholar]
  66. Morgan MJ, Ward R. 1980. Conditions for motion flow in dynamic visual noise. Vis. Res.. 205431–35
  67. Movshon JA, Newsome WT. 1996. Visual response properties of striate cortical neurons projecting to area MT in macaque monkeys. J. Neurosci. 16:237733–41 [Google Scholar]
  68. Mulligan JB, Stevenson SB, Cormack LK. 2013. Reflexive and voluntary control of smooth eye movements. Proc. SPIE, Hum. Vis. Electron. Imaging XVIII 8651:86510Z http://doi.org/10.1117/12.2010333 [Crossref] [Google Scholar]
  69. Nadler JW, Angelaki DE, DeAngelis GC. 2008. A neural representation of depth from motion parallax in macaque visual cortex. Nature 452:7187642–45 http://doi.org/10.1038/nature06814 [Crossref] [Google Scholar]
  70. Nefs HT, Harris JM. 2010. What visual information is used for stereoscopic depth displacement discrimination?. Perception 39:6727–44 [Google Scholar]
  71. Newsome WT, Paré EB. 1988. A selective impairment of motion perception following lesions of the middle temporal visual area (MT). J. Neurosci. 8:62201–11 [Google Scholar]
  72. Nilsson DE. 2006. Eyes as optical alarm systems in fan worms and ark clams. Philos. Trans. R. Soc. B 346:195–212 [Google Scholar]
  73. Nishihara HK. 1984. Practical real-time imaging stereo matcher. Opt. Eng. 23:5536–45 http://doi.org/10.1117/12.7973334 [Crossref] [Google Scholar]
  74. Norcia A, Tyler C. 1984. Temporal frequency limits for stereoscopic apparent motion processes. Vis. Res. 24:5395–401 [Google Scholar]
  75. Nover H, Anderson CH, DeAngelis GC. 2005. A logarithmic, scale-invariant representation of speed in macaque middle temporal area accounts for speed discrimination performance. J. Neurosci. 25:4310049–60 http://doi.org/10.1523/JNEUROSCI.1661-05.2005 [Crossref] [Google Scholar]
  76. Ohzawa I, DeAngelis GC, Freeman RD. 1990. Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. Science 249:49721037–41 [Google Scholar]
  77. Peng Q, Shi BE. 2010. The changing disparity energy model. Vis. Res. 50:2181–92 http://doi.org/10.1016/j.visres.2009.11.012 [Crossref] [Google Scholar]
  78. Peng Q, Shi BE. 2014. Neural population models for perception of motion in depth. Vis. Res. 101:C11–31 http://doi.org/10.1016/j.visres.2014.04.014 [Crossref] [Google Scholar]
  79. Phillipson GP, Read JCA. 2010. Stereo correspondence is optimized for large viewing distances. Eur. J. Neurosci. 32:111959–69 http://doi.org/10.1111/j.1460-9568.2010.07454.x [Crossref] [Google Scholar]
  80. Poggio GF, Motter BC, Squatrito S, Trotter Y. 1985. Responses of neurons in visual cortex (V1 and V2) of the alert macaque to dynamic random-dot stereograms. Vis. Res. 25:3397–406 [Google Scholar]
  81. Poggio GF, Talbot WH. 1981. Mechanisms of static and dynamic stereopsis in foveal cortex of the rhesus monkey. J. Physiol. 315:469–92 [Google Scholar]
  82. Ponce CR, Lomber SG, Born RT. 2008. Integrating motion and depth via parallel pathways. Nat. Neurosci. 11:2216–23 http://doi.org/10.1038/nn2039 [Crossref] [Google Scholar]
  83. Portfors-Yeomans C, Regan D. 1996. Cyclopean discrimination thresholds for the direction and speed of motion in depth. Vis. Res. 36:203265–79 [Google Scholar]
  84. Regan D. 1993. Binocular correlates of the direction of motion in depth. Vis. Res. 33:162359–60 [Google Scholar]
  85. Regan D, Beverley KI. 1979. Binocular and monocular stimuli for motion in depth: changing-disparity and changing-size feed the same motion-in-depth stage. Vis. Res. 19:121331–42 [Google Scholar]
  86. Regan D, Erkelens CJ, Collewijn H. 1986. Visual field defects for vergence eye movements and for stereomotion perception. Invest. Ophthalmol. Vis. Sci. 27:5806–19 [Google Scholar]
  87. Regan D, Gray R. 2009. Binocular processing of motion: some unresolved questions. Spatial Vis 22:11–43 http://doi.org/10.1163/156856809786618501 [Crossref] [Google Scholar]
  88. Richert M, Albright TD, Krekelberg B. 2013. The complex structure of receptive fields in the middle temporal area. Front. Syst. Neurosci. 7:2 http://doi.org/10.3389/fnsys.2013.00002 [Crossref] [Google Scholar]
  89. Rogers BJ, Bradshaw MF. 1993. Vertical disparities, differential perspective and binocular stereopsis. Nature 361:253–55 http://doi.org/10.1038/361253a0 [Crossref] [Google Scholar]
  90. Rokers B, Cormack LK, Huk AC. 2008. Strong percepts of motion through depth without strong percepts of position in depth. J. Vis. 8:46 http://doi.org/10.1167/8.4.6 [Crossref] [Google Scholar]
  91. Rokers B, Cormack LK, Huk AC. 2009. Disparity- and velocity-based signals for three-dimensional motion perception in human MT+. Nat. Neurosci. 12:81050–55 http://doi.org/10.1038/nn.2343 [Crossref] [Google Scholar]
  92. Rokers B, Czuba TB, Cormack LK, Huk AC. 2011. Motion processing with two eyes in three dimensions. J. Vis. 11:210 http://doi.org/10.1167/11.2.10 [Crossref] [Google Scholar]
  93. Roy J-P, Wurtz RH. 1990. The role of disparity-sensitive cortical neurons in signalling the direction of self-motion. Nature 348:160–62 [Google Scholar]
  94. Sanada TM, DeAngelis GC. 2014. Neural representation of motion-in-depth in area MT. J. Neurosci. 34:4715508–21 http://doi.org/10.1523/JNEUROSCI.1072-14.2014 [Crossref] [Google Scholar]
  95. Schaafsma SJ, Duysens J. 1996. Neurons in the ventral intraparietal area of awake macaque monkey closely resemble neurons in the dorsal part of the medial superior temporal area in their responses to optic flow patterns. J. Neurophysiol. 76:64056–68 [Google Scholar]
  96. Schlack A, Hoffmann K-P, Bremmer F. 2002. Interaction of linear vestibular and visual stimulation in the macaque ventral intraparietal area (VIP). Eur. J. Neurosci. 16:101877–86 [Google Scholar]
  97. Schlack A, Sterbing-D'Angelo SJ, Hartung K, Hoffmann K-P, Bremmer F. 2005. Multisensory space representations in the macaque ventral intraparietal area. J. Neurosci. 25:184616–25 http://doi.org/10.1523/JNEUROSCI.0455-05.2005 [Crossref] [Google Scholar]
  98. Shadlen MN, Newsome WT. 2001. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol. 86:41916–36 [Google Scholar]
  99. Sheliga BM, Quaia C, FitzGibbon EJ, Cumming BG. 2016. Human short-latency ocular vergence responses produced by interocular velocity differences. J. Vis. 16:1011 http://doi.org/10.1167/16.10.11 [Crossref] [Google Scholar]
  100. Shioiri S, Matsumiya K. 2009. Motion mechanisms with different spatiotemporal characteristics identified by an MAE technique with superimposed gratings. J. Vis. 9:530 [Google Scholar]
  101. Toyama K, Komatsu Y, Kasai H, Fujii K, Umetani K. 1985. Responsiveness of Clare-Bishop neurons to visual cues associated with motion of a visual stimulus in three-dimensional space. Vis. Res. 25:3407–14 [Google Scholar]
  102. Toyama K, Komatsu Y, Kozasa T. 1986. The responsiveness of Clare-Bishop neurons to motion cues for motion stereopsis. Neurosci. Res. 4:283–109 [Google Scholar]
  103. van den Berg AV, Brenner E. 1994. Why two eyes are better than one for judgements of heading. Nature 371:6499700–2 http://doi.org/10.1038/371700a0 [Crossref] [Google Scholar]
  104. Van Essen DC, Maunsell JH, Bixby JL. 1981. The middle temporal visual area in the macaque: myeloarchitecture, connections, functional properties and topographic organization. J. Comp. Neurol. 199:3293–326 http://doi.org/10.1002/cne.901990302 [Crossref] [Google Scholar]
  105. Watson AB, Ahumada AJ. 1985. Model of human visual-motion sensing. J. Opt. Soc. Am. A 2:2322–41 [Google Scholar]
  106. Wheatstone C. 1838. Contributions to the physiology of vision. Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision. Philos. Trans. R. Soc. 128:371–94 [Google Scholar]
  107. Williams DW, Sekuler R. 1984. Coherent global motion percepts from stochastic local motions. Vis. Res. 24:55–62 [Google Scholar]
  108. Zeki SM. 1974a. Cells responding to changing image size and disparity in the cortex of the rhesus monkey. J. Physiol. 242:3827–41 [Google Scholar]
  109. Zeki SM. 1974b. Functional organization of a visual area in the posterior bank of the superior temporal sulcus of the rhesus monkey. J. Physiol. 236:3549–73 [Google Scholar]
  110. Zhang T, Britten KH. 2011. Parietal area VIP causally influences heading perception during pursuit eye movements. J. Neurosci. 31:72569–75 [Google Scholar]

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error