AI for Causal Inference in Healthcare
How can we adapt large models for causal inference in healthcare, with unstructured real-world data, responsibly and efficiently?
We introduce NATURAL, a novel family of causal effect estimators built with LLMs that operate over datasets of unstructured text. Our estimators use LLM conditional distributions (over variables of interest, given the text data) to assist in the computation of classical estimators of causal effect. NATURAL estimators demonstrate remarkable performance, yielding causal effect estimates that fall within 3 percentage points of their ground truth counterparts, including on real-world Phase 3/4 clinical trials. Our results suggest that unstructured text data is a rich source of causal effect information, and NATURAL is a first step towards an automated pipeline to tap this resource.
We generalize existing worst-case frameworks to estimate the sensitivity of three common assumptions in causal inference. Empirically, worst-case conclusions about sensitivity can rely on unrealistic changes in the data-generating process. To overcome this limitation, we introduce a new criterion, the Bayesian Sensitivity Value (BSV), which computes the expected sensitivity of an estimate to assumption violations under priors constructed from real-world evidence.
Predicting the effect of interventions with many possible variations, e.g., therapeutic content that affects mental health outcomes, is invaluable, but challenging due to high-dimensionality and unseen test-time treatments. We recast causal inference as a learning problem, design objectives that directly improve causal estimation, and show how to project the effect of a high-dimensional treatment onto lower-dimensional treatment attributes, which allows a single model to answer several causal questions without additional attribute-specific training. Our experiments on Amazon Reviews demonstrate competitive performance of projected causal estimates with attribute-specific estimators.
Neural Network Function Spaces
How can we summarize a model's parameters and training data to use later without storing or iterating over the entire dataset?
We consider a specific case of approximating the function space distance (FSD) over the training set, i.e. the average distance between the outputs of two ReLU neural networks, based on approximating the architecture as a linear network with stochastic gating. Despite requiring only one parameter per unit of the network, our parametric approximation is competitive with state-of-the-art nonparametric approximations with larger memory requirements, when applied to continual learning and influence function estimation.
The Linearized Activation Function TRick (LAFTR) enabled an efficient parametric function space distance approximation for linear networks with ReLU activations. We extend this to a more general LInearized Function TRick (LIFTR) to enable data-free FSD estimation for arbitrary architectures, with particular focus on transformers. On a modular arithmetic continual learning task, a stochastic variant of LIFTR approaches oracle performance while outperforming parameter-space linearization baselines.