# Fostering the Ecosystem of AI for Social Impact Requires Expanding and Strengthening Evaluation Standards

arXiv: https://arxiv.org/abs/2510.18238 · HTML: https://mlsanity.com/paper/2510.18238/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 21 Oct 2025.

## Builds on (most cited first, 12 of 12)

- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [Difficult Lessons on Social Prediction from Wisconsin Public Schools](https://mlsanity.com/paper/2304.06205/) (2023)
- [Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects](https://mlsanity.com/paper/2111.07966/) (2021)
- [A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms](https://mlsanity.com/paper/2206.14983/) (2022)
- [Reproducibility in Machine Learning for Health](https://mlsanity.com/paper/1907.01463/) (2019)
- [Application-Driven Innovation in Machine Learning](https://mlsanity.com/paper/2403.17381/) (2024)
- [Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments](https://mlsanity.com/paper/2203.14511/) (2022)
- [Evaluating the Effectiveness of Index-Based Treatment Allocation](https://mlsanity.com/paper/2402.11771/) (2024)
- [Just Trial Once: Ongoing Causal Validation of Machine Learning Models](https://mlsanity.com/paper/2502.09467/) (2025)

## Built on by (most cited first, 2 of 2)

- [Position: Stop Preaching and Start Practising Data Frugality for Responsible Development of AI](https://mlsanity.com/paper/2602.19789/) (2026)
- [An Investigation of the NeurIPS and ICML 2025 Position Tracks](https://mlsanity.com/paper/2608.16894/) (2026)

## Abstract (arXiv)

There has been increasing research interest in AI/ML for social impact, and correspondingly more publication venues have refined review criteria for practice-driven AI/ML research. However, these review guidelines tend to most concretely recognize projects that simultaneously achieve deployment and novel ML methodological innovation. We argue that this introduces incentives for researchers that undermine the sustainability of a broader research ecosystem of social impact, which benefits from projects that make contributions on single front (applied or methodological) that may better meet project partner needs. Our position is that researchers and reviewers in machine learning for social impact must simultaneously adopt: 1) a more expansive conception of social impacts beyond deployment and 2) more rigorous evaluations of the impact of deployed systems.