mlsanityData to 14 Oct 2026

Azimuth: Systematic Error Analysis for Text Classification

arXiv 2212.08216 ↗First posted 16 Dec 2022cs.LG cs.AI cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

We present Azimuth, an open-source and easy-to-use tool to perform error analysis for text classification. Compared to other stages of the ML development cycle, such as model training and hyper-parameter tuning, the process and tooling for the error analysis stage are less mature. However, this stage is critical for the development of reliable and trustworthy AI systems. To make error analysis more systematic, we propose an approach comprising dataset analysis and model quality assessment, which Azimuth facilitates. We aim to help AI practitioners discover and address areas where the model does not generalize by leveraging and integrating a range of ML techniques, such as saliency maps, similarity, uncertainty, and behavioral analyses, all in one tool. Our code and documentation are available at github.com/servicenow/azimuth.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 21 of its 33 references that are in the map. First posted on arXiv 16 Dec 2022.

Builds on

21 of 33 references.

References · 21

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  3. 2017Billion-scale similarity search with GPUscited by 2,497
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  5. 2018Model Cards for Model Reportingcited by 1,080
  6. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  7. 2018Annotation Artifacts in Natural Language Inference Datacited by 671
  8. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  9. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  10. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 425
  11. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 395
  12. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 386
9 more
  1. 2017The (Un)reliability of saliency methodscited by 333
  2. 2021What Will it Take to Fix Benchmarking in Natural Language Understanding?cited by 203
  3. 2020A Diagnostic Study of Explainability Techniques for Text Classificationcited by 159
  4. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  5. 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
  6. 2021Robustness Gym: Unifying the NLP Evaluation Landscapecited by 108
  7. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  8. 2019AllenNLP Interpret: A Framework for Explaining Predictions of NLP Modelscited by 74
  9. 2020CrossCheck: Rapid, Reproducible, and Interpretable Model Evaluationcited by 7

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2023Making LLMs Worth Every Penny: Resource-Limited Text Classification in Bankingcited by 37
  2. 2023InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanationscited by 11
  3. 2025Fine-Tune an SLM or Prompt an LLM? The Case of Generating Low-Code Workflowscited by 6
  4. 2024Maximizing Relation Extraction Potential: A Data-Centric Study to Unveil Challenges and Opportunitiescited by 0