Machine Learning and Friends Lunch: R. Kenny Jones, Designing DSLs for 3D Shape and Scene Generation
Content
Speaker
R. Kenny Jones (Stanford University)
Abstract
Neurosymbolic methods offer a compelling framework for 3D shape and scene generation, combining the structure and controllability of programs with the flexibility of learned models. This talk investigates a central question behind that promise: how should we design the domain-specific languages that these systems rely on? I begin with scene generation methods based on expert-authored declarative and imperative DSLs, which enable structured reasoning but require manual language design. I then turn to a data-driven alternative, showing how reusable abstractions for visual programs can be discovered automatically from collections of low-level primitives. Building on that perspective, I argue that for many downstream tasks, good DSL design benefits from, and often requires, shared conceptual grounding between users and generative systems. I will present early work that explores how we might explore using LLMs as partners in the DSL design process, by constructing abstractions from high-level descriptions, exemplar shapes, and interactive concept refinement.
Speaker Bio
Kenny Jones is a postdoctoral scholar at Stanford University, where he works with Maneesh Agrawala and Jiajun Wu and is supported by a Hoffman-Yee Research Grant on Integrating Intelligence. He received his PhD in Computer Science from Brown University, where he was advised by Daniel Ritchie and supported by a Presidential Fellowship. His research explores neurosymbolic methods for understanding, representing, and generating visual data at the intersection of computer graphics, computer vision, and machine learning. Much of his recent work has focused on visual programs and domain-specific languages for applications including reverse engineering, shape and scene synthesis, and abstraction discovery.