Faculty Recruiting Support CICS

Approximate Bayesian Deep Learning for Resource-Constrained Environments

26 Jan
Wednesday, 01/26/2022 10:00am to 12:00pm
Virtual via Zoom
PhD Dissertation Proposal Defense
Speaker: Meet Vadera

Abstract: Deep learning models have shown promising results in areas including computer vision, natural language processing, speech recognition, and more. However, existing point estimation-based training methods for these models may result in predictive uncertainties that are not well calibrated, including the occurrence of confident errors. Approximate Bayesian inference methods can help to address these issues in a principled way by accounting for uncertainty in model parameters. However, these methods are computationally expensive both when computing approximations to the parameter posterior and when using an approximate parameter posterior to make predictions. They can also require significantly more storage than point estimated models.

In this thesis, we address a range of questions related to trade-offs between the quality of inference and prediction and the computational scalability of Bayesian deep learning methods. We begin by developing a framework for the comprehensive evaluation of Bayesian neural network models and applying this framework to a range of existing models and inference methods. Second, we address the problem of providing flexible trade-offs between prediction quality, run time and storage by developing and evaluating a general framework for distilling expectations with respect to the Bayesian posterior distribution of a deep neural network classifier. Third, we investigate the trade-offs between model sparsity and inference performance for deep neural network models using several approaches to deriving sparse model structures. Forth, we present a framework for correcting approximate posterior predictive distributions, encouraging them to prefer high-utility decisions. Finally, we propose the investigation of approximate Bayesian inference techniques for end-to-end object detection. In this work, we aim to specifically focus on building a probabilistic framework for vision transformer models.

Advisor: Benjamin Marlin