CIIR Talk Series: Understanding and Supporting Boundedly Rational Users in Intelligent Information Retrieval

Title: Understanding and Supporting Boundedly Rational Users in Intelligent Information Retrieval

Abstract: How users think, behave, and make decisions when interacting with information systems in tasks of varying types is a fundamental research theme in the area of interactive information retrieval (IR) and recommendation. There is substantial evidence from behavioral economics and cognitive psychology demonstrating that in the context of decision-making under uncertainty, the carriers of value behind actions are gains and losses defined relative to a reference point, rather than absolute final outcomes. This Reference Dependence Effect as a systematic cognitive bias was abstracted out of most formal models built upon unrealistic assumptions regarding human rationality. To address this gap, our research 1) investigated the effects of a series of estimated reference points on search behavior and satisfaction at both query and session levels; 2) applied the knowledge of reference dependence in predicting users'search decisions and variations in the level of satisfaction, and 3) developed novel reference-dependent evaluation metrics (ReDeMs) and meta-evaluated the performance of the metrics, in terms of correlation with user satisfaction, against that of the widely-used offline metrics on three search datasets. Our experiments demonstrate that: 1) users' search satisfaction and many aspects of search behaviors are closely associated with relative gains, losses and the associated reference points; 2) users' judgments of session-level satisfaction are significantly affected by peak and end reference moments within sessions; 3) ReDeMs integrated with a proper reference point can achieve better correlations with user satisfaction than most of the existing metrics with fine-tuned parameters. The adaptation of behavioral economics perspective enhances our understanding of user biases and related practical issues in IR and increases the explanatory power of user models by offering them a more realistic psychological foundation.

Bio: Jiqun Liu is currently an assistant professor of data science and affiliated assistant professor of psychology at the University of Oklahoma. He holds a PhD in Information Science from Rutgers University-New Brunswick. His research program focuses on the intersection of human-computer interaction (HCI), information retrieval and recommendation, and cognitive psychology. His work seeks to apply the knowledge learned about people interacting with information in user modeling, adaptive and proactive recommendation, bias-aware evaluation and intelligent nudging. His recent studies on users' bounded rationality and bias-aware intelligent information retrieval have been supported by a grant from National Science Foundation (NSF) and have been published at leading computer and data science venues. His new behavioral economics approach to user modeling, interface design, and system evaluation has also been presented in the research monograph entitled "A behavioral economics approach to interactive information retrieval: Understanding and supporting boundedly rational users", published by Springer Nature in March 2023.

To attend this talk via Zoom, click here. To obtain the passcode for this series, please see the event advertisement on the seminars email list or reach out to Alex Taubman. For any questions about this event with the Center for Intelligent Information Retrieval, please contact Jean Joyce.