mlsanityData to 7 Oct 2026

Fostering the Ecosystem of AI for Social Impact Requires Expanding and Strengthening Evaluation Standards

arXiv 2510.18238 ↗First posted 21 Oct 2025cs.LG cs.CYcited by 2

Not on the map. Why?

Abstract (arXiv)

There has been increasing research interest in AI/ML for social impact, and correspondingly more publication venues have refined review criteria for practice-driven AI/ML research. However, these review guidelines tend to most concretely recognize projects that simultaneously achieve deployment and novel ML methodological innovation. We argue that this introduces incentives for researchers that undermine the sustainability of a broader research ecosystem of social impact, which benefits from projects that make contributions on single front (applied or methodological) that may better meet project partner needs. Our position is that researchers and reviewers in machine learning for social impact must simultaneously adopt: 1) a more expansive conception of social impacts beyond deployment and 2) more rigorous evaluations of the impact of deployed systems.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 12 of its 37 references that are in the map. First posted on arXiv 21 Oct 2025.

Builds on

12 of 37 references.

References · 12

  1. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
  2. 2019Measurement and Fairnesscited by 299
  3. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  4. 2019Problem Formulation and Fairnesscited by 85
  5. 2023Difficult Lessons on Social Prediction from Wisconsin Public Schoolscited by 35
  6. 2022A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithmscited by 30
  7. 2021Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effectscited by 29
  8. 2019Reproducibility in Machine Learning for Healthcited by 24
  9. 2024Application-Driven Innovation in Machine Learningcited by 24
  10. 2022Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experimentscited by 7
  11. 2024Evaluating the Effectiveness of Index-Based Treatment Allocationcited by 5
  12. 2025Just Trial Once: Ongoing Causal Validation of Machine Learning Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Position: Stop Preaching and Start Practising Data Frugality for Responsible Development of AIcited by 0
  2. 2026An Investigation of the NeurIPS and ICML 2025 Position Trackscited by 0