Variational Inference performance strongly depends on many algorithmic choices, such as the optimization algorithm used, the objective, and the variational family. The right combination for these components is highly problem dependent, and there is currently little guidance on how to find it. This limits the use of these methods by non-expert users. The proposed work will advance Variational Inference towards an automatic and robust technology. We provide theoretically justified guiding principles for these decisions, and algorithms able to make optimal choices adaptively, creating Variational Inference based tools that are robust, flexible, and widely accessible.
Advisor: Justin Domke