PhD Thesis Defense: Mashrur Rashik, Human-Centered Design of Contextually Appropriate Conversational Agents for Enhancing Public Interaction, Engagement, and Multimodal Experiences
Content
Speaker
Abstract
In recent years, AI-powered conversational agents, or chatbots, have emerged as alternatives to traditional online surveys for eliciting information. Their ability to support natural, engaging, and personalized interactions has led to widespread adoption across domains such as healthcare, community engagement, and education. Despite significant advancements, designing conversational agents that effectively sustain user engagement and accurately interpret user intents across multiple contexts remains a challenging task. In addition, existing data collection methods often rely on static, self-directed entries, lacking interactive feedback and real-time guidance. This gap can lead to incomplete or imprecise data, limiting its usefulness in specialized domains such as healthcare and education, where high-quality data is crucial for informing personalized interventions, policy decisions, and research outcomes. This dissertation addresses these challenges by systematically investigating the design of conversational agents, specifically the role of avatars, which have a significant impact on user experience and information elicitation. It further investigates multi-agent designs, personalized interactions, and the integration of text and visual data for multimodal communication while considering application-specific factors to ensure relevant and accurate responses.
First, I explored how conversational agents can support timely data collection in domains where information spans multiple levels, such as individuals, families, cities, and urban environments. In these settings, single-agent chatbots often struggle to accurately interpret complex, multi-faceted user input and capture nuanced information across diverse topics. To address this, I investigated the use of multi-agent chatbots for multifaceted civic data collection, which reduces computational cost by distributing topics among multiple agents. In this work, I apply multi-agent chatbots to civic data collection and tackle key challenges, including selecting the right number of agents, coordinating turn-taking, maintaining smooth conversational flow across topics, and ensuring both user engagement and high-quality responses. To that end, I conducted a Wizard of Oz study to examine the design of a multi-agent chatbot for gathering public input across multiple high-level domains and their associated topics. Building on the findings, I designed and developed CommunityBots—a multi-agent chatbot platform where each chatbot agent handles a different domain individually. To manage conversation across multiple topics and chatbots, I introduced a novel Conversation and Topic Management (CTM) mechanism that handles topic-switching and chatbot-switching based on user responses and intentions. A between-subjects study with 96 crowd workers demonstrated that CommunityBots significantly increased user engagement, improved response quality, and reduced conversational interruptions compared to a single-agent baseline. The visual cues integrated into the interface helped participants better understand the functionalities of the CTM mechanism, resulting in increased user satisfaction.
Results from the CommunityBots crowd study showed that avatar visual cues improved user engagement and helped users navigate topics and chatbot switches. However, existing literature still lacks clarity on how specific avatar features affect user experience across different application contexts, device types, conversation styles, and input methods. To gain a more comprehensive understanding of the conversational agent avatar design space, I conducted a thorough analysis of existing literature to map the avatar design space. I defined a categorization of 10 dimensions that is based on the analysis and iterative coding of 266 conversational agent papers from 160 venues spanning 2003 to the present. In addition, I built an interactive browser to facilitate exploration and interaction with these dimensions and their interrelationships. This categorization lays the groundwork for researchers, designers, and practitioners to discern task-specific and contextual aspects of conversational agent avatar design.
Finally, I worked on exploring the impact of personalized interaction in conversational systems through two key applications in healthcare and climate change communication. I developed PATRIKA, an AI-enabled conversational journaling prototype designed for people with Parkinson's disease (PwPD). Traditional journaling methods, such as online surveys, often lack interactive feedback and real-time guidance, leading to incomplete or imprecise data, particularly for managing chronic conditions. To address this gap, PATRIKA incorporates cooperative conversation principles, clinical interview simulations, and personalization to create a more engaging and effective journaling experience. Through two user studies with PwPD and iterative refinements, I demonstrated that conversational journaling significantly enhances patient engagement and collects clinically valuable information. Additionally, I investigated multimodal interaction and localization through the development of CLAImate, an AI-enabled system that generates personalized and geographically tailored climate change narratives and visualizations. While personalization has shown promise in improving communication, most existing tools fail to adapt narratives and visuals to users' specific contexts and lived experiences. CLAImate addresses this gap by delivering climate narratives that integrate both text and visual data, customized to users' locations and prior climate knowledge. Findings from internal system verification, a formative study, and a pilot study demonstrate the system's effectiveness in delivering clear, informative, and context-aware climate narratives.
This dissertation advances conversational AI by showing how multi-agent chatbots, thoughtful avatar design, and personalized interactions enhance engagement and data quality in civic, healthcare, and climate domains. My proposed frameworks for managing complex, multi-topic conversations and mapping avatar design enable the development of more adaptive and context-aware agents. Furthermore, systems such as PATRIKA and CLAImate show how personalized and multimodal conversational agents can gather richer, more nuanced data based on user context and needs. Building on this work, future research can look into the scalable deployment of multimodal and personalized agents for real-world data collection and support, particularly in sensitive and high-impact applications.
Advisors
Narges Mahyar and Ali Sarvghad