Scaling Value-Aligned Learning to Robotics

Abstract: Before learning robots can be deployed in the real world, it is critical to be able to assure that their goals and behaviors will be aligned with the values of human users. While great progress has been made in the use of imitation learning to teach increasingly complex behaviors to robots, the value alignment problem has largely been ignored. The practical usefulness of such algorithms will remain limited without methods that can provide strong, finite-sample guarantees that scale to realistic robotics problems.    Toward this goal, I will first introduce a series of algorithms that challenge common, but poor, statistical assumptions commonly made in imitation learning. This work culminates in an approach that enables value-aligned imitation learning to scale to high-dimensional control problems for the first time. Second, I will discuss a body of work that leverages historically under-utilized data modalities -- ranging from human gaze and facial expressions to contact and force sensing -- to further improve the characterization of human values and environmental uncertainty, thereby minimizing risk. Taken together, these algorithms represent a significant step toward enabling value-aligned robot learning in the real world, using only modest amounts of data.   Bio: Scott Niekum is an Associate Professor and the director of the Personal Autonomous Robotics Lab (PeARL) in the Department of Computer Science at UT Austin. He is also a core faculty member in the interdepartmental robotics group at UT. His research interests include robotic manipulation, imitation learning, reinforcement learning, and human-robot interaction. Scott is a recipient of the 2018 NSF CAREER Award, the 2019 AFOSR Young Investigator Award, and the 2019 UT Austin College of Natural Sciences Teaching Excellence Award.