# From Narratives to Numbers: Valid Inference Using Language Model Predictions from Verbal Autopsy Narratives

arXiv: https://arxiv.org/abs/2404.02438 · HTML: https://mlsanity.com/paper/2404.02438/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2024.

## Built on by (most cited first, 8 of 8)

- [A Note on the Prediction-Powered Bootstrap](https://mlsanity.com/paper/2405.18379/) (2024)
- [Another look at statistical inference with machine learning-imputed data](https://mlsanity.com/paper/2411.19908/) (2024)
- [Do We Really Even Need Data? A Modern Look at Drawing Inference with Predicted Data](https://mlsanity.com/paper/2512.05456/) (2025)
- [Spatially Robust Inference with Predicted and Missing at Random Labels](https://mlsanity.com/paper/2603.11368/) (2026)
- [Unique Rashomon Sets for Robust Active Learning](https://mlsanity.com/paper/2503.06770/) (2025)
- [REALITrees: Rashomon Ensemble Active Learning for Interpretable Trees](https://mlsanity.com/paper/2603.22750/) (2026)
- [Bayesian Federated Cause-of-Death Classification and Quantification Under Distribution Shift](https://mlsanity.com/paper/2505.02257/) (2025)
- [When Surveys Become Conversations: Adaptive Matrix Validation for AI-Assisted Interviews](https://mlsanity.com/paper/2606.24244/) (2026)

## Abstract (arXiv)

In settings where most deaths occur outside the healthcare system, verbal autopsies (VAs) are a common tool to monitor trends in causes of death (COD). VAs are interviews with a surviving caregiver or relative that are used to predict the decedent's COD. Turning VAs into actionable insights for researchers and policymakers requires two steps (i) predicting likely COD using the VA interview and (ii) performing inference with predicted CODs (e.g. modeling the breakdown of causes by demographic factors using a sample of deaths). In this paper, we develop a method for valid inference using outcomes (in our case COD) predicted from free-form text using state-of-the-art NLP techniques. This method, which we call multiPPI++, extends recent work in "prediction-powered inference" to multinomial classification. We leverage a suite of NLP techniques for COD prediction and, through empirical analysis of VA data, demonstrate the effectiveness of our approach in handling transportability issues. multiPPI++ recovers ground truth estimates, regardless of which NLP model produced predictions and regardless of whether they were produced by a more accurate predictor like GPT-4-32k or a less accurate predictor like KNN. Our findings demonstrate the practical importance of inference correction for public health decision-making and suggests that if inference tasks are the end goal, having a small amount of contextually relevant, high quality labeled data is essential regardless of the NLP algorithm.