# ferret: a Framework for Benchmarking Explainers on Transformers

arXiv: https://arxiv.org/abs/2208.01575 · HTML: https://mlsanity.com/paper/2208.01575/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 54 references that are in the map.
- First posted on arXiv 2 Aug 2022.

## Builds on (most cited first, 30 of 33)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Understanding Neural Networks through Representation Erasure](https://mlsanity.com/paper/1612.08220/) (2016)
- [A Survey of the State of Explainable AI for Natural Language Processing](https://mlsanity.com/paper/2010.00711/) (2020)
- [A Diagnostic Study of Explainability Techniques for Text Classification](https://mlsanity.com/paper/2009.13295/) (2020)
- [Post-hoc Interpretability for Neural NLP: A Survey](https://mlsanity.com/paper/2108.04840/) (2021)
- [Pathologies of Neural Models Make Interpretations Difficult](https://mlsanity.com/paper/1804.07781/) (2018)
- [Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing](https://mlsanity.com/paper/2102.12060/) (2021)
- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models](https://mlsanity.com/paper/2008.05122/) (2020)
- [Robustness Gym: Unifying the NLP Evaluation Landscape](https://mlsanity.com/paper/2101.04840/) (2021)
- [Explaining NLP Models via Minimal Contrastive Editing (MiCE)](https://mlsanity.com/paper/2012.13985/) (2020)
- [AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models](https://mlsanity.com/paper/1909.09251/) (2019)
- [Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models](https://mlsanity.com/paper/1804.09299/) (2018)
- [Discretized Integrated Gradients for Explaining Language Models](https://mlsanity.com/paper/2108.13654/) (2021)
- [Evaluating and Characterizing Human Rationales](https://mlsanity.com/paper/2010.04736/) (2020)
- [On the Sensitivity and Stability of Model Interpretations in NLP](https://mlsanity.com/paper/2104.08782/) (2021)

## Built on by (most cited first, 26 of 26)

- [Inseq: An Interpretability Toolkit for Sequence Generation Models](https://mlsanity.com/paper/2302.13942/) (2023)
- [ReAGent: A Model-agnostic Feature Attribution Method for Generative Language Models](https://mlsanity.com/paper/2402.00794/) (2024)
- [Quantifying the Plausibility of Context Reliance in Neural Machine Translation](https://mlsanity.com/paper/2310.01188/) (2023)
- [Bridging the Gap in XAI-Why Reliable Metrics Matter for Explainability and Compliance](https://mlsanity.com/paper/2502.04695/) (2025)
- [FitCF: A Framework for Automatic Feature Importance-guided Counterfactual Example Generation](https://mlsanity.com/paper/2501.00777/) (2025)
- [Contrastive Learning-based Sentence Encoders Implicitly Weight Informative Words](https://mlsanity.com/paper/2310.15921/) (2023)
- [LLM Attributor: Interactive Visual Attribution for LLM Generation](https://mlsanity.com/paper/2404.01361/) (2024)
- [Explainable Compliance Detection with Multi-Hop Natural Language Inference on Assurance Case Structure](https://mlsanity.com/paper/2506.08713/) (2025)
- [Dynamic Top-k Estimation Consolidates Disagreement between Feature Attribution Methods](https://mlsanity.com/paper/2310.05619/) (2023)
- [The Role of Syntactic Span Preferences in Post-Hoc Explanation Disagreement](https://mlsanity.com/paper/2403.19424/) (2024)
- [Variationist: Exploring Multifaceted Variation and Bias in Written Language Data](https://mlsanity.com/paper/2406.17647/) (2024)
- [Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methods](https://mlsanity.com/paper/2505.01198/) (2025)
- [Clinical information extraction for Low-resource languages with Few-shot learning using Pre-trained language models and Prompting](https://mlsanity.com/paper/2403.13369/) (2024)
- [Faithfulness and the Notion of Adversarial Sensitivity in NLP Explanations](https://mlsanity.com/paper/2409.17774/) (2024)
- [Long-context Non-factoid Question Answering in Indic Languages](https://mlsanity.com/paper/2504.13615/) (2025)
- [EvalxNLP: A Framework for Benchmarking Post-Hoc Explainability Methods on NLP Models](https://mlsanity.com/paper/2505.01238/) (2025)
- [Empowering Refugee Claimants and their Lawyers: Using Machine Learning to Examine Decision-Making in Refugee Law](https://mlsanity.com/paper/2308.11531/) (2023)
- [BONES: a Benchmark fOr Neural Estimation of Shapley values](https://mlsanity.com/paper/2407.16482/) (2024)
- [Interpreto: An Explainability Library for Transformers](https://mlsanity.com/paper/2512.09730/) (2025)
- [iFlip: Iterative Feedback-driven Counterfactual Example Refinement](https://mlsanity.com/paper/2601.01446/) (2026)
- [idMotif: An Interactive Motif Identification in Protein Sequences](https://mlsanity.com/paper/2402.05953/) (2024)
- [Feature Importance Depends on Properties of the Data: Towards Choosing the Correct Explanations for Your Data and Decision Trees based Models](https://mlsanity.com/paper/2502.07153/) (2025)
- [o-MEGA: Optimized Methods for Explanation Generation and Analysis](https://mlsanity.com/paper/2510.00288/) (2025)
- [Explanation Bias is a Product: Revealing the Hidden Lexical and Position Preferences in Post-Hoc Feature Attribution](https://mlsanity.com/paper/2512.11108/) (2025)
- [Learning to Explain: Supervised Token Attribution from Transformer Attention Patterns](https://mlsanity.com/paper/2601.14112/) (2026)
- [LumiXAI: A Modular Full-Stack Framework for Feature Attribution](https://mlsanity.com/paper/2608.24524/) (2026)

## Abstract (arXiv)

As Transformers are increasingly relied upon to solve complex NLP problems, there is an increased need for their decisions to be humanly interpretable. While several explainable AI (XAI) techniques for interpreting the outputs of transformer-based models have been proposed, there is still a lack of easy access to using and comparing them. We introduce ferret, a Python library to simplify the use and comparisons of XAI methods on transformer-based classifiers. With ferret, users can visualize and compare transformers-based models output explanations using state-of-the-art XAI methods on any free-text or existing XAI corpora. Moreover, users can also evaluate ad-hoc XAI metrics to select the most faithful and plausible explanations. To align with the recently consolidated process of sharing and using transformers-based models from Hugging Face, ferret interfaces directly with its Python library. In this paper, we showcase ferret to benchmark XAI methods used on transformers for sentiment analysis and hate speech detection. We show how specific methods provide consistently better explanations and are preferable in the context of transformer models.