Faculty Recruiting Make a Gift

Machine Learning and Friends Lunch: Visual Recognition Beyond Large Labeled Training Sets

08 Feb
Thursday, 02/08/2018 12:00pm to 1:00pm
Computer Science Building, Room 150/151
Machine Learning and Friends Lunch

Abstract: The performance of recognition systems has grown by leaps and bounds these last 5 years. However, modern recognition systems still require thousands of examples per class to train. Furthermore, expanding the capabilities of the system by introducing new visual concepts again requires collecting thousands of examples for the new concept. In contrast, humans are known to quickly learn new visual concepts from as few as 1 example, and indeed require very little labeled data to build their powerful visual systems from scratch. The requirement for large training sets also makes it infeasible to use current machine vision systems for rare or hard-to-annotate visual concepts or new imaging modalities.

I will talk about some of our work on reducing this need for large labeled training sets. I will describe novel loss functions for training convolutional network-based feature representations so that new concepts can be learned from a few examples, and ways of hallucinating additional examples for data-starved classes. I will also discuss our attempt to learn feature representations without any labeled data by leveraging motion-based grouping cues. I will end with a discussion of where we are and thoughts on the way forward.


Bio: My interests are broadly in recognition in computer vision. I want to build systems that understand the visual world as well as people do. Currently, I am working on building systems that can learn about tens of thousands of visual concepts with very little or no supervision, produce rich and detailed outputs such as precise 3D shape, and reason about the world and communicate this reasoning to humans.

Before joining Cornell, I was a postdoc working with Ross Girshick, Piotr Dollar, Larry Zitnick, Laurens van der Maaten and other amazing people at Facebook AI Research. I did my PhD at beautiful Berkeley with Jitendra Malik.