Annual Review of Control, Robotics, and Autonomous Systems - Volume 5, 2022

This article is an historical overview of control theory applied to robotic manipulators, with an emphasis on the early fundamental theoretical foundations of robot control. It discusses properties of robot dynamics that enable application of advanced control methods followed by robust and adaptive control of manipulators. It also discusses nonlinear control of underactuated robots and teleoperators.

Those designing autonomous systems that interact with humans will invariably face questions about how humans think and make decisions. Fortunately, computational cognitive science offers insight into human decision-making using tools that will be familiar to those with backgrounds in optimization and control (e.g., probability theory, statistical machine learning, and reinforcement learning). Here, we review some of this work, focusing on how cognitive science can provide forward models of human decision-making and inverse models of how humans think about others’ decision-making. We highlight relevant recent developments, including approaches that synthesize black box and theory-driven modeling, accounts that recast heuristics and biases as forms of bounded optimality, and models that characterize human theory of mind and communication in decision-theoretic terms. In doing so, we aim to provide readers with a glimpse of the range of frameworks, methodologies, and actionable insights that lie at the intersection of cognitive science and control research.

Internal models are nowadays customarily used in different domains of science and engineering to describe how living organisms or artificial computational units embed their acquired knowledge about recurring events taking place in the surrounding environment. This article reviews the internal model principle in control theory, bioengineering, and neuroscience, illustrating the fundamental concepts and theoretical developments of the few last decades of research.

In this article, we provide a control-theoretic perspective on the research area of behavior trees in robotics. The key idea underlying behavior trees is to make use of modularity, hierarchies, and feedback in order to handle the complexity of a versatile robot control system. Modularity is a well-known tool to handle software complexity by enabling the development, debugging, and extension of separate modules without detailed knowledge of the entire system. A hierarchy of such modules is natural, since robot tasks can often be decomposed into a hierarchy of subtasks. Finally, feedback control is a fundamental tool for handling uncertainties and disturbances in any low-level control system, but in order to enable feedback control on the higher level, where one module decides what submodule to execute, information regarding the progress and applicability of each submodule needs to be shared in the module interfaces. We describe how these three concepts can be used in theoretical analysis, practical design, and extensions and combinations with other ideas from control theory and robotics.

An estimated 11% of adults report experiencing some form of cognitive decline, which may be associated with conditions such as stroke or dementia and can impact their memory, cognition, behavior, and physical abilities. While there are no known pharmacological treatments for many of these conditions, behavioral treatments such as cognitive training can prolong the independence of people with cognitive impairments. These treatments teach metacognitive strategies to compensate for memory difficulties in their everyday lives. Personalizing these treatments to suit the preferences and goals of an individual is critical to improving their engagement and sustainment, as well as maximizing the treatment's effectiveness. Robots have great potential to facilitate these training regimens and support people with cognitive impairments, their caregivers, and clinicians. This article examines how robots can adapt their behavior to be personalized to an individual in the context of cognitive neurorehabilitation. We provide an overview of existing robots being used to support neurorehabilitation and identify key principles for working in this space. We then examine state-of-the-art technical approaches for enabling longitudinal behavioral adaptation. To conclude, we discuss our recent work on enabling social robots to automatically adapt their behavior and explore open challenges for longitudinal behavior adaptation. This work will help guide the robotics community as it continues to provide more engaging, effective, and personalized interactions between people and robots.

This article provides a survey overview of the techniques, mechanisms, algorithms, and test and validation strategies required for the design of robotic grappling vehicles intended to approach and grapple free-flying client satellites. We concentrate on using a robotic arm to grapple a free-floating spacecraft, as distinct from spacecraft docking and berthing, where two spacecraft directly mate with each other. Robotic grappling of client spacecraft is a deceptively complex problem: It entails designing a robotic system that functions robustly in the visually stark, thermally extreme orbital environment, operating near massive and extremely expensive yet fragile client hardware, using relatively slow flight computers with limited and laggy communications. Spaceflight robotic systems are challenging to test and validate prior to deployment and extremely expensive to launch, which significantly limits opportunities to experiment with new techniques. These factors make the design and operation of orbital robotic systems significantly different from those of their terrestrial counterparts, and as a result, only a relative handful of systems have been demonstrated on orbit. Nevertheless, there is increasing interest in on-orbit robotic servicing and assembly missions, and grappling is the core requirement for these systems. Although existing systems such as the Space Station Remote Manipulator System have demonstrated extremely reliable operation, upcoming missions will attempt to expand the types of spacecraft that can be safely and dependably grappled and berthed.

The design and control of drones remain areas of active research, and here we review recent progress in this field. In this article, we discuss the design objectives and related physical scaling laws, focusing on energy consumption, agility and speed, and survivability and robustness. We divide the control of such vehicles into low-level stabilization and higher-level planning such as motion planning, and we argue that a highly relevant problem is the integration of sensing with control and planning. Lastly, we describe some vehicle morphologies and the trade-offs that they represent. We specifically compare multicopters with winged designs and consider the effects of multivehicle teams.

This article reviews approaches to controlling robots undergoing physical contact and dynamic interaction with objects in the world. Conventional motion control is compared with a hybrid combination of position and force control. Several challenges are reviewed, most importantly the problems of instability: dynamic instability due to coupling, and static instability due to exerting force. Energetically passive interactive dynamics addresses the former; a minimum stiffness proportional to the force exerted addresses the latter. Actuators, which dominate the robot's interactive dynamics, are briefly surveyed, including series elastic, variable-stiffness, and emerging designs. A comparison with human performance is made. A bioinspired approach to controlling interactive dynamics (mechanical impedance or admittance) is reviewed. Robot configuration profoundly modulates apparent inertia, whereas force feedback control has minimal influence. Superimposing first-order mechanical impedances simplifies controlling many degrees of freedom. It manages redundancy while preserving passivity (unlike null-space projection methods) and enables seamless operation into and out of singular configurations.

One potential application of multirobot systems is collective transport, a task in which multiple robots collaboratively move a payload that is too large or heavy for a single robot. In this review, we highlight a variety of control strategies for collective transport that have been developed over the past three decades. We characterize the problem scenarios that have been addressed in terms of the control objective, the robot platform and its interaction with the payload, and the robots’ capabilities and information about the payload and environment. We categorize the control strategies according to whether their sensing, computation, and communication functions are performed by a centralized supervisor or specialized robot or autonomously by the robots. We provide an overview of progress toward control strategies that can be implemented on robots with expanded autonomous functionality in uncertain environments using limited information, and we suggest directions for future work on developing such controllers.

Equivariance is a common and natural property of many nonlinear control systems, especially those associated with models of mechatronic and navigation systems. Such systems admit a symmetry, associated with the equivariance, that provides structure enabling the design of robust and high-performance observers. A key insight is to pose the observer state to lie in the symmetry group rather than on the system state space. This allows one to define a global intrinsic equivariant error but poses a challenge in defining internal dynamics for the observer. By choosing an equivariant lift of the system dynamics for the observer internal model, we show that the error dynamics have a particularly nice form. Applying the methodology of extended Kalman filtering to the equivariant error state yields a filter we term the equivariant filter. The geometry of the state-space manifold appears naturally as a curvature modification to the classical Riccati equation for extended Kalman filtering. The equivariant filter exploits the symmetry and respects the geometry of an equivariant system model, and thus yields high-performance, robust filters for a wide range of mechatronic and navigation systems.

Planning under uncertainty is critical to robotics. The partially observable Markov decision process (POMDP) is a mathematical framework for such planning problems. POMDPs are powerful because of their careful quantification of the nondeterministic effects of actions and the partial observability of the states. But for the same reason, they are notorious for their high computational complexity and have been deemed impractical for robotics. However, over the past two decades, the development of sampling-based approximate solvers has led to tremendous advances in POMDP-solving capabilities. Although these solvers do not generate the optimal solution, they can compute good POMDP solutions that significantly improve the robustness of robotics systems within reasonable computational resources, thereby making POMDPs practical for many realistic robotics problems. This article presents a review of POMDPs, emphasizing computational issues that have hindered their practicality in robotics and ideas in sampling-based solvers that have alleviated such difficulties, together with lessons learned from applying POMDPs to physical robots.

Intelligent micromachines, with dimensions ranging from a few millimeters down to hundreds of nanometers, are miniature systems capable of performing specific tasks autonomously at small scales. Enhancing the intelligence of micromachines to tackle the uncertainty and variability in complex microenvironments has applications in minimally invasive medicine, bioengineering, water cleaning, analytical chemistry, and more. Over the past decade, significant progress has been made in the construction of intelligent micromachines, evolving from simple micromachines to soft, compound, reconfigurable, encodable, multifunctional, and integrated micromachines, as well as from individual to multiagent, multiscale, hierarchical, self-organizing, and swarm micromachines. The field leverages two important trends in robotics research—the miniaturization and intelligentization of machines—but a compelling combination of these two features has yet to be realized. The core technologies required to make such tiny machines intelligent include information media, transduction, processing, exchange, and energy supply, but embedding all of these functions into a system at the micro- or nanoscale is challenging. This article offers a comprehensive introduction to the state-of-the-art technologies used to create intelligence for micromachines and provides insight into the construction of next-generation intelligent micromachines that can adapt to diverse scenarios for use in emerging fields.

Enzymes are appealing diagnostic targets because of their centrality in human health and disease. Continuous efforts spanning several decades have yielded methods for magnetically detecting the interactions of enzymes with exogenous molecular substrates. Nevertheless, measuring enzymatic activity in vivo remains challenging due to background noise, insufficient selectivity, and overlapping enzymatic functions. Magnetic micro- and nanoagents are poised to help overcome these issues by offering possible advantages such as site-selective sampling, modular architectures, new forms of magnetic detection, and favorable biocompatibility. Here, we review relevant control and detection strategies and consider examples of magnetic enzyme detection demonstrated with micro- or nanorobotic systems. Most cases have focused on proteolytic enzymes, leaving ample opportunity to expand to other classes of enzymes. Enzyme-responsive magnetic micro- and nanoagents hold promise for lowering barriers of translation and enabling preemptive, point-of-care medical applications.

Continuum robots can traverse anatomical pathways to intervene in regions deep inside the human body. They are able to steer along 3D curves in confined spaces and dexterously handle tissues. Concentric tube robots (CTRs) are continuum robots that comprise a series of precurved elastic tubes that can be translated and rotated with respect to each other to control the shape of the robot and tip pose. CTRs are a rapidly maturing technology that has seen extensive research over the past decade. Today, they are being evaluated as tools for a variety of surgical applications, as they can offer precision and manipulability in tight workspaces. This review provides an exhaustive classification of research on CTRs based on their clinical applications and highlights approaches for modeling, control, design, and sensing. Competing approaches are critically presented, leading to a discussion of future directions to address the limitations of current research and its translation to clinical applications.

Medical robotics is a rapidly advancing discipline that is leading the evolution of robot-assisted surgery, personalized rehabilitation and assistance, and hospital automation. In China, both research and commercial developments in medical robotics have undergone exponential growth in recent years. In this review, we first give an overview of the clinical and social demands that motivate the rapid development in medical robotics. For each subdiscipline (surgery, rehabilitation and personal assistance, and hospital automation), we then summarize the major research projects sponsored by National Key Research and Development Programs. The remaining technical, commercial, and regulatory challenges are highlighted. This review also outlines some of the new opportunities in endoluminal and interventional robotics, micro- and nanorobotics, soft exoskeletons, intelligent human–robot interaction, and telemedicine and telesurgery, which may support the general uptake of robotics in medicine.

The design and control of autonomous systems that operate in uncertain or adversarial environments can be facilitated by formal modeling and analysis. Probabilistic model checking is a technique to automatically verify, for a given temporal logic specification, that a system model satisfies the specification, as well as to synthesize an optimal strategy for its control. This method has recently been extended to multiagent systems that exhibit competitive or cooperative behavior modeled via stochastic games and synthesis of equilibria strategies. In this article, we provide an overview of probabilistic model checking, focusing on models supported by the PRISM and PRISM-games model checkers. This overview includes fully observable and partially observable Markov decision processes, as well as turn-based and concurrent stochastic games, together with associated probabilistic temporal logics. We demonstrate the applicability of the framework through illustrative examples from autonomous systems. Finally, we highlight research challenges and suggest directions for future work in this area.

The last half decade has seen a steep rise in the number of contributions on safe learning methods for real-world robotic deployments from both the control and reinforcement learning communities. This article provides a concise but holistic review of the recent advances made in using machine learning to achieve safe decision-making under uncertainties, with a focus on unifying the language and frameworks used in control theory and reinforcement learning research. It includes learning-based control approaches that safely improve performance by learning the uncertain dynamics, reinforcement learning approaches that encourage safety or robustness, and methods that can formally certify the safety of a learned control policy. As data- and learning-based robot control methods continue to gain traction, researchers must understand when and how to best leverage them in real-world scenarios where safety is imperative, such as when operating in close proximityto humans. We highlight some of the open challenges that will drive the field of robot learning in the coming years, and emphasize the need for realistic physics-based benchmarks to facilitate fair comparisons between control and reinforcement learning approaches.

Cyber-vulnerabilities are being exploited in a growing number of control systems. As many of these systems form the backbone of critical infrastructure and are becoming more automated and interconnected, it is of the utmost importance to develop methods that allow system designers and operators to do risk analysis and develop mitigation strategies. Over the last decade, great advances have been made in the control systems community to better understand cyber-threats and their potential impact. This article provides an overview of recent literature on secure networked control systems. Motivated by recent cyberattacks on the power grid, connected road vehicles, and process industries, a system model is introduced that covers many of the existing research studies on control system vulnerabilities. An attack space is introduced that illustrates how adversarial resources are allocated in some common attacks. The main part of the article describes three types of attacks: false data injection, replay, and denial-of-service attacks. Representative models and mathematical formulations of these attacks are given along with some proposed mitigation strategies. The focus is on linear discrete-time plant models, but various extensions are presented in the final section, which also mentions some interesting research problems for future work.

Understanding the fundamental principles and limitations of controlling complex networks is of paramount importance across natural, social, and engineering sciences. The classic notion of controllability does not capture the effort needed to control dynamical networks, and quantitative measures of controllability have been proposed to remedy this problem. This article presents an introductory overview of the practical (i.e., energy-related) aspects of controlling networks governed by linear dynamics. First, we introduce a class of energy-aware controllability metrics and discuss their properties. Then, we establish bounds on these metrics, which allow us to understand how the structure of the network impacts the control energy. Finally, we examine the problem of optimally selecting a set of control nodes so as to minimize the control effort, and compare the performance of some simple strategies to approximately solve this problem. Throughout the article, we include examples of structured and random networks to illustrate our results.

A colloidal system is a large collection of micrometer-sized particles suspended in a liquid, and the state of the system can be measured in real time, using imaging techniques and image processing. The assembly of the particles is driven by interactions between the particles and the surrounding liquid, as well as by external fields, including electromagnetic, flow, and gravitational fields. The dynamics of the many-body system are high-dimensional, nonlinear, and stochastic. However, low-order models are derived in some cases, often using physics-based order parameters, to facilitate studying the system dynamics. With an understanding of the system dynamics, and by manipulating the aforementioned interactions, one can control the assembly process in real time using open-loop and closed-loop feedback control. Theoretical studies and experimental demonstrations of colloidal self-assembly control have been reported, with methods ranging from heuristic rules to model-based optimal feedback control.