# Evaluating the Effectiveness of Index-Based Treatment Allocation

arXiv: https://arxiv.org/abs/2402.11771 · HTML: https://mlsanity.com/paper/2402.11771/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 48 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Builds on (most cited first, 13 of 13)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Collapsing Bandits and Their Application to Public Health Interventions](https://mlsanity.com/paper/2007.04432/) (2020)
- [Difficult Lessons on Social Prediction from Wisconsin Public Schools](https://mlsanity.com/paper/2304.06205/) (2023)
- [Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects](https://mlsanity.com/paper/2111.07966/) (2021)
- [Learning to Prescribe Interventions for Tuberculosis Patients Using Digital Adherence Data](https://mlsanity.com/paper/1902.01506/) (2019)
- [Q-Learning Lagrange Policies for Multi-Action Restless Bandits](https://mlsanity.com/paper/2106.12024/) (2021)
- [Scalable Decision-Focused Learning in Restless Multi-Armed Bandits with Application to Maternal and Child Health](https://mlsanity.com/paper/2202.00916/) (2022)
- [Treatment Allocation under Uncertain Costs](https://mlsanity.com/paper/2103.11066/) (2021)
- [Efficient augmentation and relaxation learning for individualized treatment rules using observational data](https://mlsanity.com/paper/1901.00663/) (2019)
- [Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments](https://mlsanity.com/paper/2203.14511/) (2022)
- [Equitable Restless Multi-Armed Bandits: A General Framework Inspired By Digital Health](https://mlsanity.com/paper/2308.09726/) (2023)
- [Improved Policy Evaluation for Randomized Trials of Algorithmic Resource Allocation](https://mlsanity.com/paper/2302.02570/) (2023)
- [Limited Resource Allocation in a Non-Markovian World: The Case of Maternal and Child Healthcare](https://mlsanity.com/paper/2305.12640/) (2023)

## Built on by (most cited first, 5 of 5)

- [Allocation Requires Prediction Only if Inequality Is Low](https://mlsanity.com/paper/2406.13882/) (2024)
- [The Value of Prediction in Identifying the Worst-Off](https://mlsanity.com/paper/2501.19334/) (2025)
- [Beyond Listenership: AI-Predicted Interventions Drive Improvements in Maternal Health Behaviours](https://mlsanity.com/paper/2507.20755/) (2025)
- [Fostering the Ecosystem of AI for Social Impact Requires Expanding and Strengthening Evaluation Standards](https://mlsanity.com/paper/2510.18238/) (2025)
- [The Hidden Cost of Waiting for Accurate Predictions](https://mlsanity.com/paper/2503.00650/) (2025)

## Abstract (arXiv)

When resources are scarce, an allocation policy is needed to decide who receives a resource. This problem occurs, for instance, when allocating scarce medical resources and is often solved using modern ML methods. This paper introduces methods to evaluate index-based allocation policies -- that allocate a fixed number of resources to those who need them the most -- by using data from a randomized control trial. Such policies create dependencies between agents, which render the assumptions behind standard statistical tests invalid and limit the effectiveness of estimators. Addressing these challenges, we translate and extend recent ideas from the statistics literature to present an efficient estimator and methods for computing asymptotically correct confidence intervals. This enables us to effectively draw valid statistical conclusions, a critical gap in previous work. Our extensive experiments validate our methodology in practical settings, while also showcasing its statistical power. We conclude by proposing and empirically verifying extensions of our methodology that enable us to reevaluate a past randomized control trial to evaluate different ML allocation policies in the context of a mHealth program, drawing previously invisible conclusions.