Annual Review of Control, Robotics, and Autonomous Systems - Volume 6, 2023

Soft robots’ flexibility and compliance give them the potential to outperform traditional rigid-bodied robots while performing multiple tasks in unexpectedly changing environments and conditions. However, soft robots are yet to reveal their full potential; nature is still far more advanced in several areas, such as locomotion and manipulation. To understand what limits their performance and hinders their transition from laboratory to real-world conditions, future studies should focus on understanding the principles behind the design and operation of soft robots. Such studies should also consider the major challenges with regard to complex materials, accurate modeling, advanced control, and intelligent behaviors. As a starting point for such studies, this review provides a current overview of the field by examining the working mechanisms of advanced actuation and sensing modalities, modeling techniques, control strategies, and learning architectures for soft robots. Next, we summarize how these approaches can be applied to create sophisticated soft robots and examine their application areas. Finally, we provide future perspectives on what key challenges should be tackled first to advance soft robotics to truly add value to our society.

Soft robotic systems are human friendly and can mimic the complex motions of animals, which introduces promising potential in various applications, ranging from novel actuation and wearable electronics to bioinspired robots operating in unstructured environments. Due to the use of soft materials, the traditional fabrication and manufacturing methods for rigid materials are unavailable for soft robots. 3D printing is a promising fabrication method for the multifunctional and multimaterial demands of soft robots, as it enables the personalization and customization of the materials and structures. This review provides perspectives on the manufacturing methods for various types of soft robotic systems and discusses the challenges and prospects of future research, including in-depth discussion of pneumatic, electrically activated, magnetically driven, and 4D-printed soft actuators and integrated soft actuators and sensors. Finally, the challenges of realizing multimaterial, multiscale, and multifunctional 3D-printed soft robots are discussed.

This article provides an exposition of the field of adaptive control and its intersections with reinforcement learning. Adaptive control and reinforcement learning are two different methods that are both commonly employed for the control of uncertain systems. Historically, adaptive control has excelled at real-time control of systems with specific model structures through adaptive rules that learn the underlying parameters while providing strict guarantees on stability, asymptotic performance, and learning. Reinforcement learning methods are applicable to a broad class of systems and are able to produce near-optimal policies for highly complex control tasks. This is often enabled by significant offline training via simulation or the collection of large input-state datasets. This article attempts to compare adaptive control and reinforcement learning using a common framework. The problem statement in each field and highlights of their results are outlined. Two specific examples of dynamic systems are used to illustrate the details of the two methods, their advantages, and their deficiencies. The need for real-time control methods that leverage tools from both approaches is motivated through the lens of this common framework.

Embodiment is a crucial concept for the autonomy and adaptivity of systems working in the physical world with high degrees of uncertainty and complexity. The physical bodies of autonomous adaptive systems heavily influence the information flow from the environment to the central processing (and vice versa), requiring us to consider the full triad of brain, body, and environment to investigate intelligent behavior. This article provides a structured review of embodied intelligence with a special emphasis on the concept of timescales and their role in self-organization and the emergence of complex behavior. We classify embodied interactions into three types—cross-timescale matching, separation, and nontemporal sequences—and discuss how these interactions were studied in the past as well as how they can contribute to the systematic investigation of complex autonomous and adaptive systems in both biological and artificial entities.

Gradient-based methods have been widely used for system design and optimization in diverse application domains. Recently, there has been a renewed interest in studying theoretical properties of these methods in the context of control and reinforcement learning. This article surveys some of the recent developments on policy optimization, a gradient-based iterative approach for feedback control synthesis that has been popularized by successes of reinforcement learning. We take an interdisciplinary perspective in our exposition that connects control theory, reinforcement learning, and large-scale optimization. We review a number of recently developed theoretical results on the optimization landscape, global convergence, and sample complexityof gradient-based methods for various continuous control problems, such as the linear quadratic regulator (LQR), control, risk-sensitive control, linear quadratic Gaussian (LQG) control, and output feedback synthesis. In conjunction with these optimization results, we also discuss how direct policy optimization handles stability and robustness concerns in learning-based control, two main desiderata in control engineering. We conclude the survey by pointing out several challenges and opportunities at the intersection of learning and control.

Sequential Monte Carlo methods—also known as particle filters—offer approximate solutions to filtering problems for nonlinear state-space systems. These filtering problems are notoriously difficult to solve in general due to a lack of closed-form expressions and challenging expectation integrals. The essential idea behind particle filters is to employ Monte Carlo integration techniques in order to ameliorate both of these challenges. This article presents an intuitive introduction to the main particle filter ideas and then unifies three commonly employed particle filtering algorithms. This unified approach relies on a nonstandard presentation of the particle filter, which has the advantage of highlighting precisely where the differences between these algorithms stem from. Some relevant extensions and successful application domains of the particle filter are also presented.

Over the past decades, robotics has shown great potential to impact the built environment, from automation to differentiation and efficient construction. However, construction processes are highly complex and require tackling a multitude of problems, from safety and robustness to ease of control and interactivity. For this reason, the field of construction robotics is still evolving, requiring finding solutions for new challenges every day. The present review analyzes the role of robotics in construction and architecture over time and highlights current trends in shifting from pure automation toward collaborative and adaptive processes that have the potential to fully integrate robotics into a rigid and challenging industry, such as construction.

Early research on physical human–robot interaction (pHRI) has necessarily focused on device design—the creation of compliant and sensorized hardware, such as exoskeletons, prostheses, and robot arms, that enables people to safely come in contact with robotic systems and to communicate about their collaborative intent. As hardware capabilities have become sufficient for many applications, and as computing has become more powerful, algorithms that support fluent and expressive use of pHRI systems have begun to play a prominent role in determining the systems’ usefulness. In this review, we describe a selection of representative algorithmic approaches that regulate and interpret pHRI, describing the progression from algorithms based on physical analogies, such as admittance control, to computational methods based on higher-level reasoning, which take advantage of multimodal communication channels. Existing algorithmic approaches largely enable task-specific pHRI, but they do not generalize to versatile human–robot collaboration. Throughout the review and in our discussion of next steps, we therefore argue that emergent embodied dialogue—bidirectional, multimodal communication that can be learned through continuous interaction—is one of the next frontiers of pHRI.

Networked control systems, where feedback loops are closed over communication networks, arise in several domains, including smart energy grids, autonomous driving, unmanned aerial vehicles, and many industrial and robotic systems active in service, production, agriculture, and smart homes and cities. In these settings, the two main layers of the system, control and communication, strongly affect each other's performance, and they also reveal the interaction between a cyber-system component, represented by information-based computing and communication technologies, and a physical-system component, represented by the environment that needs to be controlled. The information access and distribution constraints required to achieve reliable state estimation and stabilization in networked control systems have been intensively studied over the course of roughly two decades. This article reviews some of the cornerstone results in this area, draws a map for what we have learned over these years, and describes the new challenges that we will face in the future. Rather than simply listing different results, we present them in a coherent fashion using a uniform notation, and we also put them in context, highlighting both their theoreticalinsights and their practical significance. Particular attention is given to recent developments related to decentralized estimation in distributed sensing and communication systems and the information-theoretic value of event timing in the context of networked control.

This review aims to present recent developments in modeling and control of multiagent systems. A particular focus is set on crowd dynamics characterized by complex interactions among agents, also called social interactions, and large-scale systems. Specifically, in a crowd each individual agent interacts with a field generated by the other agents and the environment. These systems can be modeled at the microscopic scale by ordinary differential equations, while an alternative description at the mesoscopic scale is given by a partial differential equation for the propagation of the probability density of the agents. Control actions can be applied at the individual level as well as at the level of the corresponding fields. This article presents and compares different control types, and the specific application to multilane, multiclass traffic is developed in some detail, showing the main tools at work in a hybrid setting with relevant impacts on autonomous driving.

While noise is generally associated with uncertainties and often has a negative connotation in engineering, living organisms have evolved to adapt to (and even exploit) such uncertainty to ensure the survival of a species or implement certain functions that would have been difficult or even impossible otherwise. In this article, we review the role and impact of noise in systems and synthetic biology, with a particular emphasis on its role in the genetic control of biological systems, an area we refer to as cybergenetics. The main modeling paradigm is that of stochastic reaction networks, whose applicability goes beyond biology, as these networks can represent any population dynamics system, including ecological, epidemiological, and opinion dynamics networks. We review different ways to mathematically represent these systems, and we notably argue that the concept of ergodicity presents a particularly suitable way to characterize their stability. We then discuss noise-induced properties and show that noise can be both an asset and a nuisance in this setting. Finally, we discuss recent results on (stochastic) cybergenetics and explore their relationships to noise. Along the way, we detail the different technical and biological constraints that need to be respected when designing synthetic biological circuits. Finally, we discuss the concepts, problems, and solutions exposed in the article; raise criticisms and concerns about current ideas and approaches; suggest current (open) problems with potential solutions; and provide some ideas for future research directions.

Surface tension effects are known to be dominant at the submillimeter scale. Within this context, the literature has extensively described the underlying physics (e.g., surface tension, wetting, surface texturation, and coatings), and capillary forces have been exploited in a variety of applications (e.g., capillary picking, self-alignment, capillary sealing, and capillary bearings). As several stimuli can be used to control liquid menisci, these forces have been used mainly in microrobotics in open loop (i.e., without real-time feedback). However, at least two major sources of uncertainty hinder these forces from working properly in open loop: the variability due to contact-angle hysteresis (the difference between wetting and unwetting) and the variability in the involved volume of liquid. To be able to reject these disturbances, successful sensor integration and associated advanced control schemes need to be embedded in capillary microrobotic microsystems. This article analyzes research contributions in this field from three different perspectives: the stimulus action of the surface tension effect (light, B-field, etc.), the application field (actuation, picking, sealing, etc.), and the sensing and control schemes. Technologically complex developments coexist with elegant and straightforward engineering solutions. Biological aspects of surface tension are not included in this review.

Space-mounted robotics is becoming increasingly mainstream for many space missions. The aim of this article is threefold: first, to give a broad and quick overview of the importance of spacecraft-mounted robotics for future in-orbit servicing missions; second, to review the basic current approaches for modeling and control of spacecraft-mounted robotic systems; and third, to introduce some new developments in terms of modeling and control of spacecraft-mounted robotic manipulators using the language of hypercomplex numbers (dual quaternions). Some outstanding research questions and potential future directions in the field are also discussed.

Grasp learning has become an exciting and important topic in robotics. Just a few years ago, the problem of grasping novel objects from unstructured piles of clutter was considered a serious research challenge. Now, it is a capability that is quickly being incorporated into industrial supply chain automation. How did that happen? What is the current state of the art in robotic grasp learning, what are the different methodological approaches, and what machine learning models are used? This review attempts to give an overview of the current state of the art of grasp learning research.

Cost, efficiency, and emissions concerns have motivated the application of advanced control techniques to multiple carrier energy systems. Research in energy management and control over the last two decades has shown that significant energy and CO2 emissions reductions can be achieved. Within the last decade, this work has expanded to the domain of interconnected energy systems. The interconnection control of multiple energy carriers, conversion devices, and energy storage provides increased flexibility and energy/CO2 reduction potential. The focus of this article is on outlining the control methods required for these systems over a range of energy consumption and timescales. Dynamic interactions between multicarrier systems occur over timescales ranging from 15 minutes to seasons. The constrained nature of the resulting control problems favors optimization-based approaches.

Electric power systems are undergoing an unprecedented transition from fossil fuel–based power plants to low-inertia systems that rely predominantly on power electronics and renewable energy resources. This article reviews the resulting control challenges and modeling fallacies, at both the device and system level, and focuses on novel aspects or classical concepts that need to be revised in light of the transition to low-inertia systems. To this end, we survey the literature on modeling of low-inertia systems, review research on the control of grid-connected power converters, and discuss the frequency dynamics of low-inertia systems. Moreover, we discuss system-level services from a control perspective. Overall, we conclude that the system-theoretic mindset is essential to bridge different research communities and understand the complex interactions of power electronics, electric machines, and their controls in large-scale low-inertia power systems.

Approximately 1.1. billion people worldwide live with some form of disability, and assistive technology has the potential to increase their overall quality of life. However, the end users’ perspective and needs are often not sufficiently considered during the development of this technology, leading to frustration and nonuse of existing devices. Since its first competition in 2016, CYBATHLON has aimed to drive innovation in the field of assistive technology by motivating teams to involve end users more actively in the development process and to tailor novel devices to their actual daily-life needs. Competition tasks therefore represent unsolved daily-life challenges for people with disabilities and serve the purpose of benchmarking the latest developments from research laboratories and companies from around the world. This review describes each of the competition disciplines, their contributions to assistive technology, and remaining challenges in the user-centered development of this technology.

The Defense Advanced Research Projects Agency (DARPA) Subterranean Challenge represented a multiyear (2018–2021), competition-based initiative to inspire and shape the future of field robotics, specifically in advancing integrated technologies for operating in complex underground environments. Bringing together robotics researchers and innovators from around the world to compete in physical and simulated contests, it spotlighted significant opportunities to incentivize and extract high-value technical results and insights to inform future advances. This article captures and quantifies these results, extracts relevant insights, and offers lessons learned and recommendations for further work.