CIIR Talk Series: Piecing the Puzzle: Language Models for Multi-Document Contexts
Abstract: Large language models have shown significant capabilities in a variety of NLP tasks. Despite several advances, their abilities for processing multi-document tasks remain less explored. In this talk, I will first discuss our earlier work focusing on the extension of long-context language models for tasks that necessitate cross-document understanding, such as multi-document summarization. Subsequently, I will delve into our work on enhancing these models to manage both short and long-form generation across multiple documents. This work proposes a novel pre-training approach to improve the language models' ability to understand and integrate cross-document information. I will then present our work on extreme multi-document scenarios and the role of retrieval, providing further insights into these tasks as well as novel cross document applications in the scholarly documents. Finally I'll end by discussing challenges in evaluation and the need for new benchmarks. This talk describes joint work with Avi Cacialuru, Wen Xiao, John Giorgi, Yilun Zhao, Benjamin Newman, and several other collaborators.
Bio: Arman Cohan is an Assistant Professor of Computer Science at Yale University and a faculty Research Scientist at the Allen Institute for AI (AI2). His research spans various problems at the intersection of Natural Language Processing and Machine Learning, including Language Modeling, Representation Learning, Generation, and their applications to specialized domains include science. His research has been recognized with multiple awards, including a best paper award at EMNLP, an outstanding paper award at EACL, and an honorable mention at COLING. Prior to joining Yale, he was a Research Scientist at AI2 and an Affiliate Assistant Professor at University of Washington.
To obtain the passcode for this series, please see the event advertisement on the seminars email list or reach out to ataubman [at] cs.umass.edu (Alex Taubman).
For any questions about this event with the Center for Intelligent Information Retrieval, please contact please contact zamani [at] cs.umass.edu (Hamed Zamani).