Faculty Recruiting Support CICS

Beyond Sentences and Paragraphs, Making Progress Towards Document-Level NLP

27 Oct
Thursday, 10/27/2022 12:00pm to 1:00pm
Computer Science Building, Room 150/151, Zoom
Machine Learning and Friends Lunch
Speaker: Arman Cohan

Title: Beyond Sentences and Paragraphs, Making Progress Towards Document-Level NLP

Abstract: The field of NLP has seen tremendous recent progress towards tasks that deal with relatively short sequences of text. However, there are still notable performance gaps between model and human performance on certain tasks that require processing longer context. In this talk, I will describe a few of our works on exploring models that target NLP tasks beyond sentence and paragraph-level context. I will first briefly discuss Longformer, an efficient transformer model that can process and contextualize information across inputs of several thousands of tokens. I will then discuss how such sparse and efficient transformers can be used to address multi-document tasks. Particularly, I will discuss CDLM as an encoder-only multi-document model and PRIMERA, an encoder-decoder general pre-trained model for generation tasks. Finally, I will discuss some of our other efforts on creating challenging benchmarks to help make more progress in document-level NLP.

Bio: Arman Cohan is an incoming Assistant Professor at Yale University and a Research Scientist at the Allen Institute for AI (AI2). His broad research interest is developing natural language processing (NLP) methods towards addressing information overload particularly for language tasks that require processing significant amounts of information. This includes tasks that require document and multi-document understanding, natural language generation and summarization, as well as information discovery and filtering. His research has been recognized with multiple awards, including a best paper award at EMNLP 2017, an honorable mention at COLING 2018, and the 2019 Harold N. Glassman Distinguished Doctoral Dissertation award.

To find out more information about this event or to obtain the Zoom link, please see the event announcements from MLFL on the college email lists or contact wenlongzhao [at] cs.umass.edu (subject: MLFL%20Zoom%20Link) (Wenlong Zhao).

Host
: