mlsanityData to 7 Oct 2026

The What-If Tool: Interactive Probing of Machine Learning Models

arXiv 1907.04135 ↗First posted 9 Jul 2019cs.LG stat.MLcited by 207

Not on the map. Why?

Abstract (arXiv)

A key challenge in developing and deploying Machine Learning (ML) systems is understanding their performance across a wide range of inputs. To address this challenge, we created the What-If Tool, an open-source application that allows practitioners to probe, visualize, and analyze ML systems, with minimal coding. The What-If Tool lets practitioners test performance in hypothetical situations, analyze the importance of different data features, and visualize model behavior across multiple models and subsets of input data. It also lets practitioners measure systems according to multiple ML fairness metrics. We describe the design of the tool, and report on real-life usage at different organizations.

Key facts. Cited by 207 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 44; 2022: 42; 2023: 43; 2024: 37; 2025: 24; 2026: 17.

207 papers.

Sort:

By area:

  1. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 129
  2. 2021Robustness Gym: Unifying the NLP Evaluation Landscapecited by 107
  3. 2021If Only We Had Better Counterfactual Explanations: Five Key Deficits to Rectify in the Evaluation of Counterfactual XAI Techniquescited by 82
  4. 2023EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteriacited by 73
  5. 2022TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversationscited by 65
  6. 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
  7. 2021The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Modelscited by 63
  8. 2021NICE: An Algorithm for Nearest Instance Counterfactual Explanationscited by 62
  9. 2021Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applicationscited by 59
  10. 2021Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Makingcited by 58
  11. 2021Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanationscited by 53
  12. 2022VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformerscited by 53
38 more
  1. 2023Supporting Human-AI Collaboration in Auditing LLMs with LLMscited by 46
  2. 2021Local Explanations via Necessity and Sufficiency: Unifying Theory and Practicecited by 45
  3. 2022Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Supportcited by 37
  4. 2022Going Beyond XAI: A Systematic Survey for Explanation-Guided Learningcited by 35
  5. 2022Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkitscited by 33
  6. 2021GeCo: Quality Counterfactual Explanations in Real Timecited by 32
  7. 2022A Comprehensive Survey on Trustworthy Recommender Systemscited by 32
  8. 2023Foundation Models Meet Visualizations: Challenges and Opportunitiescited by 31
  9. 2021M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysiscited by 29
  10. 2021Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behaviorcited by 29
  11. 2022The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizationscited by 29
  12. 2024Farsight: Fostering Responsible AI Awareness During AI Application Prototypingcited by 28
  13. 2023Diagnosing Infeasible Optimization Problems Using Large Language Modelscited by 27
  14. 2023Visual Analytics For Machine Learning: A Data Perspective Surveycited by 25
  15. 2024TeachTune: Reviewing Pedagogical Agents Against Diverse Student Profiles with Simulated Studentscited by 25
  16. 2021A Sociotechnical View of Algorithmic Fairnesscited by 24
  17. 2022Explainable AI for clinical and remote health applications: a survey on tabular and time series datacited by 24
  18. 2021When Fair Ranking Meets Uncertain Inferencecited by 22
  19. 2022Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Surveycited by 22
  20. 2023Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiencescited by 21
  21. 2021Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production Systemscited by 20
  22. 2022The privacy issue of counterfactual explanations: explanation linkage attackscited by 20
  23. 2024Position: Explain to Question not to Justifycited by 20
  24. 2021Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloudcited by 19
  25. 2021Rational Shapley Valuescited by 18
  26. 2021A Framework for Fairness: A Systematic Review of Existing Fair AI Solutionscited by 18
  27. 2022Symphony: Composing Interactive Interfaces for Machine Learningcited by 18
  28. 2021Evaluating the Interpretability of Generative Models by Interactive Reconstructioncited by 16
  29. 2021Model-agnostic and Scalable Counterfactual Explanations via Reinforcement Learningcited by 15
  30. 2022Backtracking Counterfactualscited by 15
  31. 2024FairAIED: Navigating Fairness, Bias, and Ethics in Educational AI Applicationscited by 15
  32. 2021Counterfactual Explanations for Oblique Decision Trees: Exact, Efficient Algorithmscited by 14
  33. 2021Explainable Machine Learning with Prior Knowledge: An Overviewcited by 14
  34. 2023Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI Collaborationcited by 13
  35. 2024The Pursuit of Fairness in Artificial Intelligence Models: A Surveycited by 13
  36. 2021FairCanary: Rapid Continuous Explainable Fairnesscited by 12
  37. 2021VBridge: Connecting the Dots Between Features and Data to Explain Healthcare Modelscited by 12
  38. 2022A Sandbox Tool to Bias(Stress)-Test Fairness Algorithmscited by 12

50 most cited of 207 All 207 (JSON)