mlsanityData to 14 Oct 2026

Thermostat: A Large Collection of NLP Model Explanations and Analysis Tools

arXiv 2108.13961 ↗First posted 31 Aug 2021cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

In the language domain, as in other domains, neural explainability takes an ever more important role, with feature attribution methods on the forefront. Many such methods require considerable computational resources and expert knowledge about implementation details and parameter choices. To facilitate research, we present Thermostat which consists of a large collection of model explanations and accompanying analysis tools. Thermostat allows easy access to over 200k explanations for the decisions of prominent state-of-the-art models spanning across different NLP tasks, generated with multiple explainers. The dataset took over 10k GPU hours (> one year) to compile; compute time that the community now saves. The accompanying software tools allow to analyse explanations instance-wise but also accumulatively on corpus level. Users can investigate and compare models, datasets and explainers without the need to orchestrate implementation details. Thermostat is fully open source, democratizes explainability research in the language domain, circumvents redundant computations and increases comparability and replicability.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 31 Aug 2021.

Builds on

39 of 59 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  10. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  11. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  12. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
27 more
  1. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  2. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  3. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 294
  4. 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
  5. 2016Understanding Neural Networks through Representation Erasurecited by 224
  6. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  7. 2020Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?cited by 190
  8. 2015Visualizing and Understanding Recurrent Networkscited by 160
  9. 2020A Diagnostic Study of Explainability Techniques for Text Classificationcited by 159
  10. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  11. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  12. 2020Evaluating Explanations: How much do explanations from the teacher aid students?cited by 94
  13. 2017Explaining Recurrent Neural Network Predictions in Sentiment Analysiscited by 88
  14. 2020How does this interaction affect me? Interpretable attribution for feature interactionscited by 73
  15. 2020On quantitative aspects of model interpretabilitycited by 71
  16. 2019Incorporating Priors with Feature Attribution on Text Classificationcited by 68
  17. 2020How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Maskingcited by 63
  18. 2016Explaining Predictions of Non-Linear Classifiers in NLPcited by 37
  19. 2021Evaluating Saliency Methods for Neural Language Modelscited by 37
  20. 2021Order in the Court: Explainable AI Methods Prone to Disagreementcited by 36
  21. 2021Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processingcited by 30
  22. 2019Human-grounded Evaluations of Explanation Methods for Text Classificationcited by 28
  23. 2020To what extent do human explanations of model behavior align with actual model behavior?cited by 18
  24. 2021On the Lack of Robust Interpretability of Neural Text Classifierscited by 17
  25. 2021Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Maskscited by 12
  26. 2021Investigating sanity checks for saliency maps with image and text classificationcited by 11
  27. 2021Efficient Explanations from Empirical Explainerscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 4; 2024: 1; 2025: 0; 2026: 1.

11 papers.

  1. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  2. 2022ferret: a Framework for Benchmarking Explainers on Transformerscited by 26
  3. 2023MDACE: MIMIC Documents Annotated with Code Evidencecited by 19
  4. 2022A Song of (Dis)agreement: Evaluating the Evaluation of Explainable Artificial Intelligence in Natural Language Processingcited by 13
  5. 2022Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methodscited by 13
  6. 2022Mediators: Conversational Agents Explaining NLP Model Behaviorcited by 7
  7. 2022An Attention Matrix for Every Decision: Faithfulness-based Arbitration Among Multiple Attention-Based Interpretations of Transformers in Text Classificationcited by 6
  8. 2023Decoding Layer Saliency in Language Transformerscited by 5
  9. 2024LLM Attributor: Interactive Visual Attribution for LLM Generationcited by 5
  10. 2023Efficient Shapley Values Estimation by Amortization for Text Classificationcited by 4
  11. 2026Explaining Black-Box Language Models: Learning to Optimize Linguistically-Structured Word Subsetscited by 0