# Deployment of AI-Assisted Interventions: Capacity Constraints and Noisy Compliance

arXiv: https://arxiv.org/abs/2604.14370 · HTML: https://mlsanity.com/paper/2604.14370/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Builds on (most cited first, 25 of 25)

- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Scalable and accurate deep learning for electronic health records](https://mlsanity.com/paper/1801.07860/) (2018)
- [Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities](https://mlsanity.com/paper/2307.13565/) (2023)
- [Decision-Focused Learning: Through the Lens of Learning to Rank](https://mlsanity.com/paper/2112.03609/) (2021)
- [Difficult Lessons on Social Prediction from Wisconsin Public Schools](https://mlsanity.com/paper/2304.06205/) (2023)
- [Experimental Evaluation of Individualized Treatment Rules](https://mlsanity.com/paper/1905.05389/) (2019)
- [Dynamic optimization with side information](https://mlsanity.com/paper/1907.07307/) (2019)
- [Cost-Effective Incentive Allocation via Structured Counterfactual Inference](https://mlsanity.com/paper/1902.02495/) (2019)
- [On Data-Driven Prescriptive Analytics with Side Information: A Regularized Nadaraya-Watson Approach](https://mlsanity.com/paper/2110.04855/) (2021)
- [A Deployed Online Reinforcement Learning Algorithm In An Oral Health Clinical Trial](https://mlsanity.com/paper/2409.02069/) (2024)
- [Efficient Policy Learning from Surrogate-Loss Classification Reductions](https://mlsanity.com/paper/2002.05153/) (2020)
- [Treatment Allocation under Uncertain Costs](https://mlsanity.com/paper/2103.11066/) (2021)
- [Equity and Artificial Intelligence in Education: Will "AIEd" Amplify or Alleviate Inequities in Education?](https://mlsanity.com/paper/2104.12920/) (2021)
- [Policy Optimization for Personalized Interventions in Behavioral Health](https://mlsanity.com/paper/2303.12206/) (2023)
- [Learning treatment effects while treating those in need](https://mlsanity.com/paper/2407.07596/) (2024)
- [Replicable Bandits for Digital Health Interventions](https://mlsanity.com/paper/2407.15377/) (2024)
- [Qini Curves for Multi-Armed Treatment Rules](https://mlsanity.com/paper/2306.11979/) (2023)
- [Optimization over Continuous and Multi-dimensional Decisions with Observational Data](https://mlsanity.com/paper/1807.04183/) (2018)
- [Learning to Call: A Field Trial of a Collaborative Bandit Algorithm for Improved Message Delivery in Mobile Maternal Health](https://mlsanity.com/paper/2507.16356/) (2025)
- [Beyond Listenership: AI-Predicted Interventions Drive Improvements in Maternal Health Behaviours](https://mlsanity.com/paper/2507.20755/) (2025)
- [Negative Spillover: A Potential Source of Bias in Pragmatic Clinical Trials](https://mlsanity.com/paper/2309.10978/) (2023)
- [Operational Dosage: Implications of Capacity Constraints for the Design and Interpretation of Experiments](https://mlsanity.com/paper/2407.21322/) (2024)
- [Evaluating Prediction-based Interventions with Human Decision Makers In Mind](https://mlsanity.com/paper/2503.05704/) (2025)
- [Discretion in the Loop: Human Expertise in Algorithm-Assisted College Advising](https://mlsanity.com/paper/2505.13325/) (2025)
- [Early Warning Index for Patient Deteriorations in Hospitals](https://mlsanity.com/paper/2512.14683/) (2025)

## Abstract (arXiv)

AI tools increasingly drive targeted interventions in various service settings, including healthcare, education, and public services. Algorithms score individuals, trigger outreach to those above a threshold (e.g., high-risk or high-value), and encourage them to request service; then providers deliver service to those who request. Much of the work in this area has focused on improving predictive accuracy, implicitly assuming that better predictions lead to better outcomes. We show that predictions are only one component of a larger service system: when service capacity is limited and behavioral responses to outreach are probabilistic, system efficacy depends on operational forces that predictive accuracy does not capture. In such settings, the optimal score threshold must balance two effects: ensuring all capacity is filled (utilization) and, when capacity is constrained, preventing low-value requests from crowding out high-value ones (cannibalization). We characterize the optimal threshold and prove that thresholds based solely on predictive accuracy are generally suboptimal. Further, algorithm selection metrics such as AUC can be misaligned with operational performance: they weight all thresholds equally, while optimal deployment uses a subset of thresholds that depends on both capacity and compliance behavior. We introduce a new metric, Operational AUC (OpAUC), and show that it identifies the efficacy-optimal algorithm. Finally, we conduct a case study on sepsis early warning data that illustrates the magnitude of the improvement available from better threshold selection and shows that a predictor with lower AUC can achieve higher system efficacy under optimal deployment.