CIIR Talk Series: What ails multihop QA and how to fix it.

Title: What ails multihop QA and how to fix it.

Abstract: Multihop QA has seen much empirical progress on many datasets recently. However, training and evaluating on typical crowdsourced datasets is problematic because of the potential for shortcut reasoning based on artifacts. What can we do about this? In this three part talk, I will first show how we can formalize and measure disconnected reasoning, a type of bad multihop reasoning. We devise an automatic transform of an existing dataset that allows us to both measure and design incentives to reduce disconnected reasoning. In the second part, I will discuss constructing new datasets using a bottom-up construction process, where we can join singlehop questions to form multihop questions. I will show how this process allows us to better control for desired properties in the resulting dataset. In the third part, I will briefly present our new work which shows how synthetically generated data can be used to teach a broad range of multihop skills in a reliable manner. 

Bio: Niranjan is an Assistant Professor in the Computer Science department at Stony Brook University, where he heads the Language Understanding and Reasoning lab (LUNR). Prior to joining Stony Brook, he was a post-doctoral researcher in the University of Washington, and was one of the early members of the Allen Institute for Artificial Intelligence. Niranjan completed his PhD in Computer Science from the University of Massachusetts Amherst. 

When he is not writing abstracts, he fancies painting them. He believes his abilities for one are marginally better than the other.

To attend this talk via Zoom, click here. To obtain the passcode for this series, please see the event advertisement on the seminars email list or reach out to Alex Taubman. For any questions about this event with the Center for Intelligent Information Retrieval, please contact Hamed Zamani.