mlsanityData to 7 Oct 2026

Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems

arXiv 1806.07552 ↗First posted 20 Jun 2018cs.AIcited by 47

Not on the map. Why?

Abstract (arXiv)

Several researchers have argued that a machine learning system's interpretability should be defined in relation to a specific agent or task: we should not ask if the system is interpretable, but to whom is it interpretable. We describe a model intended to help answer this question, by identifying different roles that agents can fulfill in relation to the machine learning system. We illustrate the use of our model in a variety of scenarios, exploring how an agent's role influences its goals, and the implications for defining interpretability. Finally, we make suggestions for how our model could be useful to interpretability researchers, system developers, and regulatory bodies auditing machine learning systems.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 9; 2023: 7; 2024: 10; 2025: 4; 2026: 6.

47 papers.

Sort:

By area:

  1. 2021Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needscited by 64
  2. 2022"Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interactioncited by 63
  3. 2021How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanationscited by 58
  4. 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systemscited by 58
  5. 2022Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contextscited by 53
  6. 2021The Who in XAI: How AI Background Shapes Perceptions of AI Explanationscited by 48
  7. 2024Trust in AI: Progress, Challenges, and Future Directionscited by 35
  8. 2023Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAIcited by 33
  9. 2021Question-Driven Design Process for Explainable AI User Experiencescited by 31
  10. 2026The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systemscited by 27
  11. 2021Fast Hierarchical Games for Image Explanationscited by 15
  12. 2021Directive Explanations for Actionable Explainability in Machine Learning Applicationscited by 14
35 more
  1. 2022From Black Boxes to Conversations: Incorporating XAI in a Conversational Agentcited by 14
  2. 2023A Comprehensive Review on Financial Explainable AIcited by 10
  3. 2024How should AI decisions be explained? Requirements for Explanations from the Perspective of European Lawcited by 10
  4. 2021How to choose an Explainability Method? Towards a Methodical Implementation of XAI in Practicecited by 9
  5. 2021AI Explainability 360: Impact and Designcited by 8
  6. 2022A Survey on XAI for 5G and Beyond Security: Technical Aspects, Challenges and Research Directionscited by 8
  7. 2022On the Safety of Interpretable Machine Learning: A Maximum Deviation Approachcited by 7
  8. 2021Explainable Artificial Intelligence (XAI): An Engineering Perspectivecited by 6
  9. 2022SHAP-XRT: The Shapley Value Meets Conditional Independence Testingcited by 6
  10. 2024What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blockscited by 6
  11. 2021On Two XAI Cultures: A Case Study of Non-technical Explanations in Deployed AI Systemcited by 5
  12. 2022A Means-End Account of Explainable Artificial Intelligencecited by 5
  13. 2021Seven challenges for harmonizing explainability requirementscited by 4
  14. 2024Regulation Games for Trustworthy Machine Learningcited by 3
  15. 2024Sufficient and Necessary Explanations (and What Lies in Between)cited by 3
  16. 2024A Mechanistic Explanatory Strategy for XAIcited by 3
  17. 2023The Case Against Explainabilitycited by 2
  18. 2023DCNFIS: Deep Convolutional Neuro-Fuzzy Inference Systemcited by 2
  19. 2024Understanding with toy surrogate models in machine learningcited by 2
  20. 2025Explainable AI Systems Must Be Contestable: Here's How to Make It Happencited by 2
  21. 2025On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomycited by 2
  22. 2022Explaining Any ML Model? -- On Goals and Capabilities of XAIcited by 1
  23. 2022The Influence of Explainable Artificial Intelligence: Nudging Behaviour or Boosting Capability?cited by 1
  24. 2023Flexible and Inherently Comprehensible Knowledge Representation for Data-Efficient Learning and Trustworthy Human-Machine Teaming in Manufacturing Environmentscited by 1
  25. 2023Towards Fair and Explainable AI using a Human-Centered AI Approachcited by 1
  26. 2024A look under the hood of the Interactive Deep Learning Enterprise (No-IDLE)cited by 1
  27. 2025A Taxonomy for Design and Evaluation of Prompt-Based Natural Language Explanationscited by 1
  28. 2026What is Missing? Explaining Neurons Activated by Absent Conceptscited by 1
  29. 2023Five policy uses of algorithmic transparency and explainabilitycited by 0
  30. 2024The Explabox: Model-Agnostic Machine Learning Transparency & Analysiscited by 0
  31. 2024The Role of XAI in Transforming Aeronautics and Aerospace Systemscited by 0
  32. 2026Scientific Theory of a Black-Box: A Life Cycle-Scale XAI Framework Based on Constructive Empiricismcited by 0
  33. 2026Transferable XAI: Relating Understanding Across Domains with Explanation Transfercited by 0
  34. 2026An Abstract Architecture for Explainable Autonomy in Hazardous Environmentscited by 0
  35. 2026Dynamic Entanglement-Weighted Pruning for Quantum Federated Unlearning in Supply-Chain Risk Predictioncited by 0