Title: Reconstructing and Tracking 3D Humans from Video
Abstract: The majority of the visual content we capture and have access to focuses on humans. In order to build AI systems that can understand and learn from this content, it is crucial to enable perception of humans from these visual observations. Despite the advances of 2D perception, we ultimately want our systems to also allow for the 3D perception of humans.
In this talk, I will present our recent works on automatic perception of 3D humans from images and videos. First, I will explore the use of large scale movie sequences to train systems for 3D human reconstruction. Despite the rich nature of movie data (i.e., large variety of behaviors and interactions, long range video), the fragmentation of movies in "shots" can complicate their analysis. Instead of treating this fragmentation as noise, I will describe how we leverage this multi-shot continuity as supervisory signal for 3D reconstruction. Then, I will discuss our recent work that uses the resulting 3D human reconstruction models to lift people to 3D and then track them over video sequences. I will identify the importance of extracting rich 3D human representations and highlight the benefits over traditional tracking on the 2D image plane.
Bio: Georgios Pavlakos is a Postdoctoral Researcher at UC Berkeley, advised by Angjoo Kanazawa and Jitendra Malik. His research interests include computer vision, machine learning, and robotics. He completed his PhD in Computer Science at the University of Pennsylvania with his advisor, Kostas Daniilidis. He has spent time at Max Planck Institute with Michael Black and at Facebook Reality Labs. His PhD dissertation received the Morris and Dorothy Rubinoff Award for the Best Computer Science Dissertation at UPenn.
To obtain the Zoom link for this event, please see the event announcements from MLFL on the college email lists or contact wenlongzhao [at] cs.umass.edu (subject: MLFL%20Zoom%20Link) (Wenlong Zhao).
If you are interested in a 1:1 meeting with Georgios, please sign up on the meeting sheet here. Meeting with the speaker individually is a great chance to dive deeper into the speaker's research, share your own research and get feedback, and potentially start a collaboration.
Students who are interested in a 1:1 meeting with Georgios should check the link in their email.