On Cautious Interaction

Abstract: As long as a model appears to describe the relevant aspects of the world satisfactorily, we may continue, cautiously, to use it; when it fails to do so, we need to search for a better one. In particular, any causal understandings that we may feel we have attained must always be treated as tentative and subject to revision should further observation of the world require it. — Philip Dawid, Causal Inference Without Counterfactuals (2000).

This talk focuses on the theme of cautious interaction, which, using the language of Philip Dawid can be explicated through three important questions: (1) what does it mean to use a model cautiously; (2) what does it mean for a model to fail to adequately describe the world; and (3) what does it mean to search for a better one? So called, “model free,” Deep Reinforcement Learning (DRL) is the framework though which we will discuss this theme, because it affords a general framework though which we can explore these three questions. Specifically, we will discuss this theme in relation to the ongoing work, “ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages,” which shows that incorporating modifications linked to cautious interaction improves DRL methods with respect to overall performance and generalization.

Bio: Andrew Jesson is an incoming Postdoc at Columbia University working with David Blei and Donald Green on Machine Learning for Large Scale Field Experiments for the Social Sciences. Andrew has previously worked with Yarin Gal at the Oxford Theoretical and Applied Machine Learning group and is set to receive his Doctor of Philosophy from the University of Oxford. The focus of his doctoral research has been on understanding and developing methods for scalable structural and statistical uncertainty in the estimation of conditional average treatment effects from observational data, with applications to active learning and experimental design. Andrew worked as a program manager at Imagia where he helped develop machine learning methodology for medical imaging applications, and he completed is Masters of Electrical Engineering at McGill University under the supervision of Tal Arbel.

For questions, please contact wenlongzhao [at] cs.umass.edu (Wenlong Zhao).