mlsanityData to 14 Oct 2026

Responsiveness Verification: Will Predictions Change? How Much? How Often?

arXiv 2507.02169 ↗First posted 2 Jul 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Machine learning models are often used in applications where their inputs change due to routine interactions, strategic manipulation, or noise. In such settings, models can undermine safety as these changes can lead them to predict over regions of the input space they have not seen. We propose to address these safety risks by measuring responsiveness---the probability that a model output attains a target prediction under feasible interactions defined by the interaction model. We develop algorithms to estimate responsiveness for any machine learning model, and a framework to specify broad classes of interaction models. We pair these algorithms with statistical guarantees that support practical validation. We demonstrate how our tools can promote safety and reliability across domains by detecting preclusion in recidivism prediction, estimating the cost of gaming in content moderation, and testing the robustness of benchmarks for LLMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 65 references that are in the map. First posted on arXiv 2 Jul 2025.

Builds on

38 of 65 references.

References · 38

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  6. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  7. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  8. 2017Counterfactual Fairnesscited by 804
  9. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 614
  10. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
  11. 2020Performative Predictioncited by 327
  12. 2015Strategic Classificationcited by 257
26 more
  1. 2020Algorithmic Recourse: from Counterfactual Explanations to Interventionscited by 250
  2. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 249
  3. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  4. 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 205
  5. 2019Predictive Multiplicity in Classificationcited by 166
  6. 2017Strategic Classification from Revealed Preferencescited by 128
  7. 2019The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasonscited by 115
  8. 2020Stochastic Optimization for Performative Predictioncited by 93
  9. 2019Strategic Classification is Causal Modeling in Disguisecited by 92
  10. 2021Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Testscited by 77
  11. 2018The Disparate Effects of Strategic Manipulationcited by 75
  12. 2017The spread of low-credibility content by social botscited by 73
  13. 2021Strategic Classification in the Darkcited by 66
  14. 2021Strategic Classification Made Practicalcited by 59
  15. 2015Interpretable Classification Models for Recidivism Predictioncited by 44
  16. 2022Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recoursecited by 40
  17. 2015Monotonic Calibrated Interpolated Look-Up Tablescited by 35
  18. 2024Phi-3 Safety Post-Training: Aligning Language Models with a "Break-Fix" Cyclecited by 32
  19. 2023Engagement, User Satisfaction, and the Amplification of Divisive Content on Social Mediacited by 30
  20. 2020Learning Models for Actionable Recoursecited by 18
  21. 2020Adversarial Feature Selection against Evasion Attackscited by 13
  22. 2023Distributionally Robust Recourse Actioncited by 12
  23. 2024Feature Responsiveness Scores: Model-Agnostic Explanations for Recoursecited by 8
  24. 2023Prediction without Preclusion: Recourse Verification with Reachable Setscited by 4
  25. 2024Improving the Robustness and Clinical Applicability of Automatic Respiratory Sound Classification Using Deep Learning-Based Audio Enhancement: Algorithm Development and Validationcited by 1
  26. 2025Understanding Fixed Predictions via Confined Regionscited by 1

Built on by

Not cited yet.