Faculty Recruiting Support CICS

Natural Language Processing for Lexical Corpus Analysis

23 Apr
Friday, 04/23/2021 10:00am to 12:00pm
Zoom Meeting
PhD Thesis Defense
Speaker: Abe Handler

Zoom Meeting: https://umass-amherst.zoom.us/j/96656828176?pwd=dDJESGF0Z3JPdmxQUHByRTVIZ2VQQT09
Meeting ID: 966 5682 8176
Passcode: 675571

People have been analyzing documents by reading keywords in context for centuries. Traditional approaches like paper concordances or digital keyword-in-context viewers display all occurrences of a single word from a corpus vocabulary amid immediately surrounding tokens or characters, to show readers how individual lexical items are used in bodies of text. We propose that these common tools are one particular application of a more general approach to analyzing documents, which we define as lexical corpus analysis. We then propose new natural language processing techniques for lexically-focused corpus investigation, and demonstrate how such methods can be used to create new user-facing tools for analyzing corpora.

Our contributions are divided into three parts. In Part 1, we consider how to represent a corpus lexicon to best reflect human mental and linguistic models of a domain, and propose a natural language processing (NLP) method for enriching a unigram corpus vocabulary with multiword phases. In Part 2, we consider how lexical systems might show query terms in context to best satisfy user search need, and offer several new extractive techniques focused on summarizing mentions of a query term in context. Finally, in Part 3, we apply our proposed NLP methods towards new user-facing systems for lexical corpus analysis, and present user studies with journalists and historians which investigate how new lexical tools can help such users in their work.

Advisor: Brendan O'Connor