Machine Learning and Friends Lunch: Ryan Louie, Adapting LLMs to Upskill Novices for High-Stakes Domains
Content
Speaker
Ryan Louie (Stanford University)
Abstract
Communities today suffer from demand-capacity problems in high-stakes service domains—such as mental health counseling—where demand for people skilled to provide essential services far exceeds supply. Can LLMs upskill novices, when expert supervision is in short supply, and ultimately scale access to human-led services in high-stakes domains? To adapt LLM-augmented training systems to empower novices, we must overcome several core technical challenges: (1) limited access to ML datasets due to privacy and ethical constraints; (2) need for outputs that meet the quality standards of domain-experts whose time to provide feedback is limited. This talk presents my research developing CARE, an AI system for psychotherapy skills training—as a key domain in which I've addressed these technical challenges. I will highlight two methods that power efficient adaptation of LLMs for skill training in mental health: Roleplay-doh, a pipeline that converts expert critiques into Constitutional AI principles to create realistic patient simulations (EMNLP’24), and a co-designed feedback system that mirrors how senior therapists supervise novices, enhanced through self-improvement techniques that reduce clinically inappropriate AI suggestions (ACL’24). Then, I will share how we’ve evaluated the educational benefits of LLM-simulated practice and feedback through conducting a randomized lab study with 90+ novice counselors. I’ll end by discussing remaining questions for adapting LLM-augmented training systems to empower people to become more effective in high-stakes domains like mental health care.
Speaker Bio
Ryan Louie is a postdoctoral researcher in the Computer Science Department at Stanford, affiliated with the Stanford NLP Group, Stanford HCI Group, and Stanford AI Lab (SAIL). He develops NLP and HCI systems that model domain-expertise and structure effective interactions between novices and AI to realize the potential of novices to acquire skills and use them to support community well-being. Ryan completed his PhD from Northwestern University’s Technology and Social Behavior program in 2023 and was awarded a Google PhD Fellowship in 2022.