Machine Learning and Friends Lunch (Online)

Title: "A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores"

 

Abstract: The increased use of algorithmic predictions in sensitive domains has been accompanied by both enthusiasm and concern. To understand the opportunities and risks of these technologies, it is key to study how experts alter their decisions when using such tools. In this paper, we study the adoption of an algorithmic tool used to assist child maltreatment hotline screening decisions. We focus on the question: Are humans capable of identifying cases in which the machine is wrong, and of overriding those recommendations? We first show that humans do alter their behavior when the tool is deployed. Then, we show that humans are less likely to adhere to the machine's recommendation when the score displayed is an incorrect estimate of risk, even when overriding the recommendation requires supervisory approval. These results highlight the risks of full automation and the importance of designing decision pipelines that provide humans with autonomy.

 

Bio: Maria De-Arteaga is an Assistant Professor at the Information, Risk and Operations Management Department at the University of Texas at Austin, where she is also a core faculty member of the Machine Learning Laboratory. She received a joint PhD in Machine Learning and Public Policy from Carnegie Mellon University. Her research focuses on the risks and opportunities of using machine learning for decision support in high-stakes settings. Her work has been awarded the Best Thematic Paper Award at NAACL’19, the Innovation Award on Data Science at Data for Policy’16, and has been featured by UN Women and Global Pulse in their report Gender Equality and Big Data: Making Gender Data Visible. She is a recipient of a 2020 Google Award for Inclusion Research, a 2018 Microsoft Research Dissertation Grant, and was named an EECS 2019 Rising Star. In 2017 she co-founded the Machine Learning for the Developing World (ML4D) Workshop series at NeurIPS.

 

To obtain the Zoom link for this event, please see the event announcements from MLFL on the college email lists or contact kalpesh [at] cs.umass.edu (subject: MLFL) (Kalpesh Krishna).