PhD Thesis Defense: Pracheta Amaranath, Causal Inference for Simulation and Simulation for Causal Inference
Content
Speaker:
Abstract:
Simulation models stand in for systems or processes on which direct experimentation is prohibitively expensive, slow or impossible. What distinguishes a simulator from other predictive models is that it is a causal model: it posits an explicit mechanism by which the state of the system changes, and allows for interventions on that mechanism. This thesis investigates the interplay between simulation and causal inference, studying two very different uses of simulators as causal models: how causal modeling can improve approximations of simulation models, and how simulation models can improve evaluation of causal inference algorithms.
In the first part, Causal Inference for Simulation, we develop modular dynamic Bayesian networks (MDBNs) as causal metamodels for discrete-event simulations of queueing networks. Conventional metamodels typically map simulation inputs to a single aggregate output, so a metamodel fitted for one query cannot be used for other queries. MDBNs instead approximate the temporal evolution of the simulation system states while preserving their underlying causal mechanisms, so that a single trained metamodel efficiently answers a range of probabilistic and causal queries on the simulator. Using an M/M/1 queue and a network of Markovian queues as running examples, we demonstrate how to specify the structure of MDBNs, learn its parameters and run efficient inference algorithms on them. We then extend this representation to a non-Markovian G/M/1 queue by representing the general interarrival process with a Markovian approximation, namely a phase-type distribution. To keep MDBNs tractable on larger queueing networks, we introduce a nonparametric representation of the MDBN parameters and a corresponding inference algorithm that scales with the size of the network.
In the second part, Simulation for Causal Inference, we examine the use of simulation models as tools to improve the evaluation of causal inference algorithms. Evaluating causal inference algorithms is a challenging task due to the lack of counterfactual outcomes or ground-truth data, invalidating the use of standard techniques such as cross-validation. In these situations, generative neural networks fitted to a real dataset serve as simulation models that produce synthetic datasets with known outcomes. An empirical evaluation of existing generative neural network methods reveals that small sample sizes, architectural biases, and fixed parameter specifications produce inconsistent datasets and unreliable estimator evaluations. To address these issues, we introduce simulation-based inference for causal evaluation (SBICE), a Bayesian framework that models simulator parameters as uncertain and infers a posterior distribution over the configurations compatible with the observed data. This approach supports principled sensitivity analyses and improves the robustness of causal estimator evaluation.
Advisors:
David Jensen and Peter Haas