Annual Review of Vision Science - Volume 10, 2024

Everybody loves illusions. At times, the content on the internet seems to be mostly about illusions—shoes, dresses, straight lines looking bent. This attraction has a long history. Almost 2,000 years ago, Ptolemy marveled at how the sail of a distant boat could appear convex or concave. This sense of marvel continues to drive our fascination with illusions; indeed, few other corners of science can boast of such a large reach. However, illusions not only draw in the crowds; they also offer insights into visual processes. This review starts with a simple definition of illusions as conflicts between perception and cognition, where what we see does not agree with what we believe we should see. This mismatch can be either because cognition has misunderstood how perception works or because perception has misjudged the visual input. It is the perceptual errors that offer the chance to track the development of perception across visual regions. Unfortunately, the effects of illusions in different brain regions cannot be isolated in any simple way: Top-down projections from attention broadcast the expected perceptual properties everywhere, obscuring the critical evidence of where the illusion and perception emerge. The second part of this review then highlights the roadblocks to research raised by attention and describes current solutions for accessing what illusions can offer.

Sighted animals use visual signals to discern directional motion in their environment. Motion is not directly detected by visual neurons, and it must instead be computed from light signals that vary over space and time. This makes visual motion estimation a near universal neural computation, and decades of research have revealed much about the algorithms and mechanisms that generate directional signals. The idea that sensory systems are optimized for performance in natural environments has deeply impacted this research. In this article, we review the many ways that optimization has been used to quantitatively model visual motion estimation and reveal its underlying principles. We emphasize that no single optimization theory has dominated the literature. Instead, researchers have adeptly incorporated different computational demands and biological constraints that are pertinent to the specific brain system and animal model under study. The successes and failures of the resulting optimization models have thereby provided insights into how computational demands and biological constraints together shape neural computation.

The ventral visual pathway transforms retinal images into neural representations that support object understanding, including exquisite appreciation of precise 2D pattern shape and 3D volumetric shape. We articulate a framework for understanding the goals of this transformation and how they are achieved by neural coding at successive ventral pathway stages. The critical goals are (a) radical compression to make shape information communicable across axonal bundles and storable in memory, (b) explicit coding to make shape information easily readable by the rest of the brain and thus accessible for cognition and behavioral control, and (c) representational stability to maintain consistent perception across highly variable viewing conditions. We describe how each transformational step in ventral pathway vision serves one or more of these goals. This three-goal framework unifies discoveries about ventral shape processing into a neural explanation for our remarkable experience of shape as a vivid, richly detailed aspect of the natural world.

Our visual systems are remarkably adept at deriving the shape and material properties of surfaces even when only one image of a surface is available. This ability implies that a single image of a surface contains potent information about both surface shape and material. However, from a computational perspective, the problem of deriving surface shape and material is formally ill posed. Any given image could be due to many combinations of shape, material, and illumination. Early computational models required prior knowledge about two of the three scene variables to derive the third. However, such models are biologically implausible because our visual systems are tasked with extracting all relevant scene variables from images simultaneously. This review describes recent progress in understanding how the visual system solves this problem by identifying complex forms of image structure that support its ability to simultaneously derive the shape and material properties of surfaces from images.

Inferences made about objects via vision, such as rapid and accurate categorization, are core to primate cognition despite the algorithmic challenge posed by varying viewpoints and scenes. Until recently, the brain mechanisms that support these capabilities were deeply mysterious. However, over the past decade, this scientific mystery has been illuminated by the discovery and development of brain-inspired, image-computable, artificial neural network (ANN) systems that rival primates in these behavioral feats. Apart from fundamentally changing the landscape of artificial intelligence, modified versions of these ANN systems are the current leading scientific hypotheses of an integrated set of mechanisms in the primate ventral visual stream that support core object recognition. What separates brain-mapped versions of these systems from prior conceptual models is that they are sensory computable, mechanistic, anatomically referenced, and testable (SMART). In this article, we review and provide perspective on the brain mechanisms addressed by the current leading SMART models. We review their empirical brain and behavioral alignment successes and failures, discuss the next frontiers for an even more accurate mechanistic understanding, and outline the likely applications.

Animals live in visually complex environments. As a result, visual systems have evolved mechanisms that simplify visual processing and allow animals to focus on the information that is most relevant to adaptive decision making. This review explores two key mechanisms that animals use to efficiently process visual information: categorization and specialization. Categorization occurs when an animal's perceptual system sorts continuously varying stimuli into a set of discrete categories. Specialization occurs when particular classes of stimuli are processed using distinct cognitive operations that are not used for other classes of stimuli. We also describe a nonadaptive consequence of simplifying heuristics: visual illusions, where visual perception consistently misleads the viewer about the state of the external world or objects within it. We take an explicitly comparative approach by exploring similarities and differences in visual cognition across human and nonhuman taxa. Considering areas of convergence and divergence across taxa provides insight into the evolution and function of visual systems and associated perceptual strategies.

What are the core learning algorithms in brains? Nativists propose that intelligence emerges from innate domain-specific knowledge systems, whereas empiricists propose that intelligence emerges from domain-general systems that learn domain-specific knowledge from experience. We address this debate by reviewing digital twin studies designed to reverse engineer the learning algorithms in newborn brains. In digital twin studies, newborn animals and artificial agents are raised in the same environments and tested with the same tasks, permitting direct comparison of their learning abilities. Supporting empiricism, digital twin studies show that domain-general algorithms learn animal-like object perception when trained on the first-person visual experiences of newborn animals. Supporting nativism, digital twin studies show that domain-general algorithms produce innate domain-specific knowledge when trained on prenatal experiences (retinal waves). We argue that learning across humans, animals, and machines can be explained by a universal principle, which we call space-time fitting. Space-time fitting explains both empiricist and nativist phenomena, providing a unified framework for understanding the origins of intelligence.

This article reviews nearly 60 years of solid-state image sensor evolution and identifies potential new frontiers in the field. From early work in the 1960s, through the development of charge-coupled device image sensors, to the complementary metal oxide semiconductor image sensors now ubiquitous in our lives, we discuss highlights in the evolutionary chain. New frontiers, such as 3D stacked technology, photon-counting technology, and others, are briefly discussed.

The study of biological optics would be complicated enough if light only came in a single wavelength. However, altering the wavelength (or distribution of wavelengths) of light has multiple effects on optics, including on diffraction, scattering (of various sorts), transmission through and reflection by various media, fluorescence, and waveguiding properties, among others. In this review, we consider just one wavelength-dependent optical effect: longitudinal chromatic aberration (LCA). All vertebrate eyes that have been tested have significant LCA, with shorter (bluer) wavelengths of light focusing closer to the front of the eye than longer (redder) wavelengths. We consider the role of LCA in the visual system in terms of both how it could degrade visual acuity and how biological systems make use of it.

Animal models are critical for understanding the initiation and progression of myopia, a refractive condition that causes blurred distance vision. The prevalence of myopia is rapidly increasing worldwide, and myopia increases the risk of developing potentially blinding diseases. Current pharmacological, optical, and environmental interventions attenuate myopia progression in children, but it is still unclear how this occurs or how these interventions can be improved to increase their protective effects. To optimize myopia interventions, directed mechanistic studies are needed. The mouse model is well-suited to these studies because of its well-characterized visual system and the genetic experimental tools available, which can be combined with pharmacological and environmental manipulations for powerful investigations of causation. This review describes aspects of the mouse visual system that support its use as a myopia model and presents genetic, pharmacological, and environmental studies that significantly contribute to our understanding of the mechanisms that underlie myopigenesis.

The fovea is a highly specialized region of the central retina, defined by an absence of inner retinal layers and the accompanying vasculature, an increased density of cone photoreceptors, a near absence of rod photoreceptors, and unique private-line photoreceptor to midget ganglion cell circuitry. These anatomical specializations support high-acuity vision in humans. While direct study of foveal shape and size is routinely performed using optical coherence tomography, examination of the other anatomical specializations of the fovea has only recently become possible using an array of adaptive optics (AO)-based imaging tools. These devices correct for the eye's monochromatic aberrations and permit cellular-resolution imaging of the living retina. In this article, we review the application of AO-based imaging techniques to conditions affecting the fovea, with an emphasis on how imaging has advanced our understanding of pathophysiology.

The retina is an ideal model for understanding the fundamental rules for how neural networks are constructed. The compact neural networks of the retina perform all of the initial processing of visual information before transmission to higher visual centers in the brain. The field of retinal connectomics uses high-resolution electron microscopy datasets to map the intricate organization of these networks and further our understanding of how these computations are performed by revealing the fundamental topologies and allowable networks behind retinal computations. In this article, we review some of the notable advances that retinal connectomics has provided in our understanding of the specific cells and the organization of their connectivities within the retina, as well as how these are shaped in development and break down in disease. Using these anatomical maps to inform modeling has been, and will continue to be, instrumental in understanding how the retina processes visual signals.

The continuous function of vertebrate photoreceptors requires regeneration of their visual pigment following its destruction upon activation by light (photobleaching). For rods, the chromophore required for the regeneration of rhodopsin is derived from the adjacent retinal pigmented epithelium (RPE) cells through a series of reactions collectively known as the RPE visual cycle. Mounting biochemical and functional evidence demonstrates that, for cones, pigment regeneration is supported by the parallel supply with chromophore by two pathways—the canonical RPE visual cycle and a second, cone-specific retina visual cycle that involves the Müller glial cells in the neural retina. In this article, we review historical information that led to the discovery of the retina visual cycle and discuss what is currently known about the reactions and molecular components of this pathway and its functional role in supporting cone-mediated vision.

Carotenoids, yellow and red pigments found abundantly in nature, play essential roles in various aspects of human physiology. They serve as critical molecules in vision by functioning as antioxidants and as filters for blue light within the retina. Furthermore, carotenoids are the natural precursors of vitamin A, which is indispensable for the synthesis of retinaldehyde, the visual chromophore, and retinoic acid, a small molecule that regulates gene expression. Insufficient levels of carotenoids and retinoids have been linked to age-related macular degeneration and xerophthalmia, respectively. Nevertheless, the mechanisms by which the eye maintains carotenoid and retinoid homeostasis have remained a mystery. Recent breakthroughs identified the molecular players involved in this process and provided valuable biochemical insights into their functioning. Mutations in the corresponding genes disrupt the homeostasis of carotenoids and retinoids, leading to visual system pathologies. This review aims to consolidate our current understanding of these pathways, including their regulatory principles.

The sense of vision begins in the retina, where light is detected and processed through a complex series of synaptic connections into meaningful information relayed to the brain via retinal ganglion cells. Light responses begin as tonic and graded signals in photoreceptors, later emerging from the retina as a series of spikes from ganglion cells. Processing by the retina extracts critical features of the visual world, including spatial frequency, temporal frequency, motion direction, color, contrast, and luminance. To achieve this, the retina has evolved specialized and unique synapse types. These include the ribbon synapses of photoreceptors and bipolar cells, the dendritic synapses of amacrine and horizontal cells, and unconventional synaptic feedback from horizontal cells to photoreceptors. We review these unique synapses in the retina with a focus on the presynaptic molecules and physiological properties that shape their capabilities.

Synapse formation within the retinal circuit ensures that distinct neuronal types can communicate efficiently to process visual signals. Synapses thus form the core of the visual computations performed by the retinal circuit. Retinal synapses are diverse but can be broadly categorized into multipartner ribbon synapses and 1:1 conventional synapses. In this article, we review our current understanding of the cellular and molecular mechanisms that regulate the functional establishment of mammalian retinal synapses, including the role of adhesion proteins, synaptic proteins, extracellular matrix and cytoskeletal-associated proteins, and activity-dependent cues. We outline future directions and areas of research that will expand our knowledge of these mechanisms. Understanding the regulators moderating synapse formation and function not only reveals the integrated developmental processes that establish retinal circuits, but also divulges the identity of mechanisms that could be engaged during disease and degeneration.

The success of genome-wide association studies (GWASs) in uncovering genetic variants associated with complex eye diseases has paved the way for the development of risk prediction approaches based on disease genetics. Derived from GWAS data, polygenic risk scores (PRSs) have been emerging as a promising indicator of an individual's genetic liability to disease. In this review, we recap the current progress of PRS development and utility across a range of common eye diseases. While illustrating the prediction accuracy of PRSs and their valuable role in risk stratification for certain eye diseases, we also address PRSs’ uncertain implementation in clinical settings at this stage, particularly in circumstances where limited treatment options are available. Finally, we discuss obstacles in translating PRSs into practice, including barriers to clinical impact, issues when working with different ancestry groups, and communicating risk scores, as well as projections for future improvements.

Glaucoma, a leading cause of irreversible blindness, is characterized by the progressive loss of retinal ganglion cells (RGCs) and subsequent visual field defects. RGCs, as the final output neurons of the retina, perform key computations underpinning human pattern vision, such as contrast coding. Conventionally, glaucoma has been associated with peripheral vision loss, and thus, relatively little attention has been paid to deficits in central vision. However, recent advancements in retinal imaging techniques have significantly bolstered research into glaucomatous damage of the macula, revealing that it is prevalent even in the early stages of glaucoma. Thus, it is an opportune time to explore how glaucomatous damage undermines the perceptual processes associated with central visual function. This review showcases recent studies addressing central dysfunction in the early and moderate stages of glaucoma. It further emphasizes the need to characterize glaucomatous damage in both central and peripheral vision, as they jointly affect an individual's everyday activities.

Geographic atrophy (GA), the non-neovascular advanced form of age-related macular degeneration, remains an important disease area in which treatment needs are currently unmet. Recent clinical trials using drugs that target the complement pathway have shown modest yet consistent reductions in GA expansion but without commensurate changes in measures of visual function. In this review, we summarize information from the wide range of studies describing the characteristics of GA morphology and enumerate the factors influencing the growth rates of lesions and the directionality of expansion. In addition, we review the relationship between GA growth and the various measures of vision that reflect changes in function. We consider the reasons for the discordance between the anatomical and functional endpoints in current use and discuss methods to align these key outcomes.

Central serous chorioretinopathy (CSCR) is the fourth most common medical retinal disease. Moderate vision loss occurs in approximately one-third of patients who have the chronic form of the disease. CSCR has a multifactorial etiology, with acquired risk factors and increasing evidence of genetic susceptibility factors. The detection of new gene variants in CSCR and association of these variants with age-related macular degeneration provide insights into possible disease mechanisms. The contribution of multimodal ocular imaging and associated research studies to the modern-day clinical investigation of CSCR has been significant. This review aims to provide an overview of the most significant epidemiological and genetic studies of CSCR, in addition to describing its clinical and multimodal imaging features. The review also provides an update of the latest evidence from studies investigating pathophysiological mechanisms in CSCR and current opinions on multimodal imaging to better classify this complex retinal disease.