PhD Dissertation Proposal: Yian Wang, Physics-Grounded Synthetic Data Generation for Scalable Robotic Manipulation
Content
Speaker:
Abstract:
Learning-based robotic manipulation requires data spanning diverse objects, environments, physical interactions, and task intents. Synthetic generation offers a scalable route to this coverage, but scale alone does not guarantee utility: a trajectory may obey simulated dynamics while arising from an implausible scene, expressing the wrong function, or specifying an outcome that a robot cannot execute. This proposal therefore treats synthetic data generation as an integrated research stack. Useful synthetic data must be grounded in task-relevant physics, populated with semantically meaningful and interaction-ready content, and converted into physically executable robot supervision.
The dissertation develops this thesis through three chapters, each addressing a necessary layer of the stack. Chapter~I addresses physical coverage: because manipulation extends beyond rigid objects, it develops simulation platforms for diverse materials specifically designed for robot manipulation, supporting optimization, policy learning, and real-to-sim calibration. Chapter~II addresses the content bottleneck: even an accurate simulator remains limited in task coverage without sufficiently diverse objects and environments, so it combines generative models with geometric and physical reasoning to create interactive scenes, functional assets, articulated objects, and stable physical arrangements. Chapter~III addresses the supervision bottleneck: generated environments and arrangements define interaction settings but do not by themselves provide robot actions or training data, so it develops autonomous systems that turn semantic task specifications and generated content into validated demonstrations, trajectories, policies, and embodiments. Together, these chapters establish practical tools and general design principles for generating manipulation data that are physically trustworthy, semantically useful, and directly actionable by robots.
Advisor:
Chuang Gan