# Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud

arXiv: https://arxiv.org/abs/2109.03285 · HTML: https://mlsanity.com/paper/2109.03285/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 57 references that are in the map.
- First posted on arXiv 7 Sep 2021.

## Builds on (most cited first, 30 of 34)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Empirical Risk Minimization under Fairness Constraints](https://mlsanity.com/paper/1802.08626/) (2018)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Fairness of Exposure in Rankings](https://mlsanity.com/paper/1802.07281/) (2018)
- [Feature relevance quantification in explainable AI: A causal problem](https://mlsanity.com/paper/1910.13413/) (2019)
- [Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search](https://mlsanity.com/paper/1905.01989/) (2019)
- [The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons](https://mlsanity.com/paper/1912.04930/) (2019)
- [Model Reconstruction from Model Explanations](https://mlsanity.com/paper/1807.05185/) (2018)
- [Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI](https://mlsanity.com/paper/2005.05906/) (2020)
- [FlipTest: Fairness Testing via Optimal Transport](https://mlsanity.com/paper/1906.09218/) (2019)
- [Average Individual Fairness: Algorithms, Generalization and Experiments](https://mlsanity.com/paper/1905.10607/) (2019)

## Built on by (most cited first, 19 of 19)

- [Fairness in Recommendation: Foundations, Methods and Applications](https://mlsanity.com/paper/2205.13619/) (2022)
- [Amazon SageMaker Model Monitor: A System for Real-Time Insights into Deployed Machine Learning Models](https://mlsanity.com/paper/2111.13657/) (2021)
- [OxonFair: A Flexible Toolkit for Algorithmic Fairness](https://mlsanity.com/paper/2407.13710/) (2024)
- [A Human-Centric Take on Model Monitoring](https://mlsanity.com/paper/2206.02868/) (2022)
- [Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools](https://mlsanity.com/paper/2309.17337/) (2023)
- [Implications of the AI Act for Non-Discrimination Law and Algorithmic Fairness](https://mlsanity.com/paper/2403.20089/) (2024)
- [Measuring What Matters: Connecting AI Ethics Evaluations to System Attributes, Hazards, and Harms](https://mlsanity.com/paper/2510.10339/) (2025)
- [ICON2: Reliably Benchmarking Predictive Inequity in Object Detection](https://mlsanity.com/paper/2306.04482/) (2023)
- [Machine Learning practices and infrastructures](https://mlsanity.com/paper/2307.06518/) (2023)
- [Data augmentation and explainability for bias discovery and mitigation in deep learning](https://mlsanity.com/paper/2308.09464/) (2023)
- [EdnaML: A Declarative API and Framework for Reproducible Deep Learning](https://mlsanity.com/paper/2211.06783/) (2022)
- [ComplAI: Theory of A Unified Framework for Multi-factor Assessment of Black-Box Supervised Machine Learning Models](https://mlsanity.com/paper/2212.14599/) (2022)
- [Mapping the Potential of Explainable AI for Fairness Along the AI Lifecycle](https://mlsanity.com/paper/2404.18736/) (2024)
- [FairSense-AI: Responsible AI Meets Sustainability](https://mlsanity.com/paper/2503.02865/) (2025)
- [Statistical Inference and Learning for Shapley Additive Explanations (SHAP)](https://mlsanity.com/paper/2602.10532/) (2026)
- [Constructive Interpretability with CoLabel: Corroborative Integration, Complementary Features, and Collaborative Learning](https://mlsanity.com/paper/2205.10011/) (2022)
- [DispaRisk: Auditing Fairness Through Usable Information](https://mlsanity.com/paper/2405.12372/) (2024)
- [Ratio1 -- AI meta-OS](https://mlsanity.com/paper/2509.12223/) (2025)
- [Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Study](https://mlsanity.com/paper/2512.15791/) (2025)

## Abstract (arXiv)

Understanding the predictions made by machine learning (ML) models and their potential biases remains a challenging and labor-intensive task that depends on the application, the dataset, and the specific model. We present Amazon SageMaker Clarify, an explainability feature for Amazon SageMaker that launched in December 2020, providing insights into data and ML models by identifying biases and explaining predictions. It is deeply integrated into Amazon SageMaker, a fully managed service that enables data scientists and developers to build, train, and deploy ML models at any scale. Clarify supports bias detection and feature importance computation across the ML lifecycle, during data preparation, model evaluation, and post-deployment monitoring. We outline the desiderata derived from customer input, the modular architecture, and the methodology for bias and explanation computations. Further, we describe the technical challenges encountered and the tradeoffs we had to make. For illustration, we discuss two customer use cases. We present our deployment results including qualitative customer feedback and a quantitative evaluation. Finally, we summarize lessons learned, and discuss best practices for the successful adoption of fairness and explanation tools in practice.