PhD Thesis Defense: Zonghai Yao, Long-Horizon Health AI Agents That Think and Act
Content
Speaker:
Abstract:
Health care unfolds through linked decisions rather than isolated responses. We study two research programs. Clinical Decision Support combines findings across medical images, gathers missing information, searches medical sources, and produces a reviewable record. Patient Education explains clinical information, checks and repairs understanding, connects imaging findings in clinical notes to marked images and language, and adapts support across sessions. We study long-horizon agents because each action can change the information, decision, or understanding available at a later step. They think by interpreting available information and deciding what is needed, and act by asking, searching, documenting, explaining, checking, or revising.
Our Clinical Decision Support studies showed the main difficulty was using distributed evidence in realistic diagnosis. In difficult multi-image cases, image interpretation caused more errors than later medical reasoning; providing an expert imaging summary produced much larger gains than adding longer reasoning. When decisive clues were removed, models often still answered instead of abstaining. We address these failures by organizing training around linked case images, designing agents that proactively inquire about missing history, examinations, and tests while updating a differential diagnosis, and searching authoritative websites and literature when the case is insufficient. We use clinician corrections to train agents to remove unsupported content and restore missing facts in notes.
In our Patient Education studies, we move from presenting information to checking whether it can guide the patient's next action. We explain medical terms within the source record, connect discharge instructions into a usable plan, and use focused questions and corrective feedback to locate misunderstandings. For imaging findings in clinical notes, we mark the relevant view with clinician-style circles, arrows, or other cues and pair it with a plain-language explanation instead of text alone. Across chronic-disease visits, agents often repeated broad information after a patient's barrier changed or outside resistance appeared. We then develop memory and planning methods that use prior goals, rejected suggestions, new events, and unresolved concerns to choose the next response.
Together, these projects show that useful agents connect each response to the evidence, patient understanding, and prior interaction that the next step requires.
Advisor:
Hong Yu