Faculty Recruiting Support CICS

CIIR Talk Series

12 Feb
Friday, 02/12/2021 1:00pm to 2:00pm
Virtual via Zoom
Special Event

Title: Online Hyperparameter Optimization by Real-time Recurrent Learning

Abstract: Conventional hyperparameter optimization methods are computationally intensive and hard to generalize to scenarios that require dynamically adapting hyperparameters, such as life-long learning. Here, we propose an online hyperparameter optimization algorithm that is asymptotically exact and computationally tractable, both theoretically and practically. Our framework takes advantage of the analogy between hyperparameter optimization and parameter learning in recurrent neural networks (RNNs). It adapts a well-studied family of online learning algorithms for RNNs to tune hyperparameters and network parameters simultaneously, without repeatedly rolling out iterative optimization. This procedure yields systematically better generalization performance compared to standard methods, at a fraction of wallclock time.

Bio:  Kyunghyun Cho is an associate professor of computer science and data science at New York University and CIFAR Fellow of Learning in Machines & Brains. He was a research scientist at Facebook AI Research from June 2017 to May 2020 and a postdoctoral fellow at University of Montreal until Summer 2015 under the supervision of Prof. Yoshua Bengio, after receiving PhD and MSc degrees from Aalto University April 2011 and April 2014, respectively, under the supervision of Prof. Juha Karhunen, Dr. Tapani Raiko and Dr. Alexander Ilin. He tries his best to find a balance among machine learning, natural language processing, and life, but almost always fails to do so.

To attend this talk via Zoom, click here. Participants will need a passcode to attend this event. If you need the passcode for this series, please see the event advertisement on the seminars email list or reach out to Alex Taubman. For any other questions about this event with the Center for Intelligent Information Retrieval, please contact Jean Joyce.