Machine Learning and Friends Lunch: Wei-Chiu Ma, Towards Physically-Grounded Digital Twins and Beyond
Content
Speaker: Wei-Chiu Ma (Cornell University)
Abstract: Generative AI and foundation models have revolutionized numerous fields (e.g., vision, NLP), transforming our lives in many ways. However, their impact on robotics remains relatively limited compared to other domains. One critical hurdle preventing robotics from reaching the "GPT moment" is the lack of sufficient data. Unlike the abundant image and text data available on the web, real-world robotic data is much more scarce. Collecting this data is expensive, time-consuming, and, most importantly, presents significant safety concerns.
In this context, the automatic creation of realistic, interactable, and highly detailed virtual replicas of physical environments offers immense potential. By making digital twins look real and act real, we can use them as dynamic, virtual testbeds for training and evaluating robotic agents at scale. In this talk, I will share our recent progress in advancing digital twin construction and how it enables more robust policy learning. By building replicas that are not only visually and geometrically accurate but also physically grounded, robotic agents deployed in these mirror worlds can interact with their environments and leverage observations and feedback to learn decision-making policies that transfer seamlessly to their real-world counterparts -- safely and at scale.
Bio: Wei-Chiu Ma is an Assistant Professor of Computer Science at Cornell University. His research lies at the intersection of 3D/4D computer vision and robotics, with a focus on building AI systems that can understand, reconstruct, and re-simulate the dynamic world. Wei-Chiu is a recipient of the Siebel Scholarship and was selected as a rising star in Cyber Physical Systems. His work has been covered by media outlets such as WIRED, DeepLearning.AI, MIT News, etc. Previously, Wei-Chiu was a Sr. Research Scientist at UberATG and Waabi, where he served as the technical lead of the sensor simulation team. His contribution to autonomy and simulation have led to 15+ patents. He received his Ph.D. in EECS from MIT and his M.S. in Robotics from CMU.