mlsanityData to 14 Oct 2026

Inseq: An Interpretability Toolkit for Sequence Generation Models

arXiv 2302.13942 ↗First posted 27 Feb 2023cs.CL cs.AI cs.HCcited by 78

Not on the map. Why?

Abstract (arXiv)

Past work in natural language processing interpretability focused mainly on popular classification tasks while largely overlooking generation settings, partly due to a lack of dedicated tools. In this work, we introduce Inseq, a Python library to democratize access to interpretability analyses of sequence generation models. Inseq enables intuitive and optimized extraction of models' internal information and feature importance scores for popular decoder-only and encoder-decoder Transformers architectures. We showcase its potential by adopting it to highlight gender biases in machine translation models and locate factual knowledge inside GPT-2. Thanks to its extensible interface supporting cutting-edge techniques such as contrastive feature attribution, Inseq can drive future advances in explainable natural language generation, centralizing good practices and enabling fair and reproducible model evaluations.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). Builds on 69 of its 87 references that are in the map. First posted on arXiv 27 Feb 2023.

Builds on

69 of 87 references.

References · 69

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  9. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  11. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  12. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
38 more
  1. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  2. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  3. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  4. 2018Sanity Checks for Saliency Mapscited by 1,156
  5. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  6. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  7. 2019Attention is not Explanationcited by 929
  8. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  9. 2020Quantifying Attention Flow in Transformerscited by 775
  10. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  11. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  12. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  13. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  14. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 530
  15. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  16. 2018Gender Bias in Coreference Resolutioncited by 442
  17. 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
  18. 2019Gradio: Hassle-Free Sharing and Testing of ML Models in the Wildcited by 178
  19. 2020A Diagnostic Study of Explainability Techniques for Text Classificationcited by 159
  20. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 156
  21. 2018Pathologies of Neural Models Make Interpretations Difficultcited by 140
  22. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 138
  23. 2020The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MTcited by 128
  24. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  25. 2018Assessing Gender Bias in Machine Translation -- A Case Study with Google Translatecited by 121
  26. 2021Gender Bias in Machine Translationcited by 117
  27. 2021Guided Integrated Gradients: An Adaptive Path Method for Removing Noisecited by 102
  28. 2018Context-Aware Neural Machine Translation Learns Anaphora Resolutioncited by 97
  29. 2022Interpreting Language Models with Contrastive Explanationscited by 92
  30. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  31. 2020Analyzing the Source and Target Contributions to Predictions in Neural Machine Translationcited by 71
  32. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  33. 2021Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translationcited by 62
  34. 2019On Measuring Gender Bias in Translation of Gender-neutral Pronounscited by 60
  35. 2017A causal framework for explaining the predictions of black-box sequence-to-sequence modelscited by 58
  36. 2021Incorporating Residual and Normalization Layers into Analysis of Masked Language Modelscited by 51
  37. 2021Discretized Integrated Gradients for Explaining Language Modelscited by 46
  38. 2022Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformercited by 45

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 17; 2024: 27; 2025: 22; 2026: 12.

78 papers.

Sort:

By area:

  1. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  2. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  3. 2023Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Modelscited by 118
  4. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 46
  5. 2023Explaining How Transformers Use Context to Build Predictionscited by 43
  6. 2023Using Captum to Explain Generative Language Modelscited by 33
  7. 2023Open-Source LLMs for Text Annotation: A Practical Guide for Model Setting and Fine-Tuningcited by 32
  8. 2024Backward Lens: Projecting Language Model Gradients into the Vocabulary Spacecited by 32
  9. 2024MambaLRP: Explaining Selective State Space Sequence Modelscited by 32
  10. 2025Steering MoE LLMs via Expert (De)Activationcited by 26
  11. 2024Peering into the Mind of Language Models: An Approach for Attribution in Contextual Question Answeringcited by 25
  12. 2024Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generationcited by 25
38 more
  1. 2024XAI meets LLMs: A Survey of the Relation between Explainable AI and Large Language Modelscited by 23
  2. 2023A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translationcited by 19
  3. 2023Demystifying Embedding Spaces using Large Language Modelscited by 17
  4. 2023TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documentscited by 17
  5. 2024LLMCheckup: Conversational Examination of Large Language Models via Interpretability Tools and Self-Explanationscited by 16
  6. 2024LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Modelscited by 16
  7. 2023Are Character-level Translations Worth the Wait? Comparing ByT5 and mT5 for Machine Translationcited by 15
  8. 2024What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Studycited by 15
  9. 2025Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associationscited by 15
  10. 2023Assessing the Reliability of Large Language Model Knowledgecited by 13
  11. 2025LLM Jailbreak Detection for (Almost) Free!cited by 13
  12. 2024ReAGent: A Model-agnostic Feature Attribution Method for Generative Language Modelscited by 12
  13. 2024Evaluating Human Alignment and Model Faithfulness of LLM Rationalecited by 12
  14. 2024Enhancing Post-Hoc Attributions in Long Document Comprehension via Coarse Grained Answer Decompositioncited by 12
  15. 2023InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanationscited by 11
  16. 2024Multi-Level Explanations for Generative Language Modelscited by 11
  17. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  18. 2023RAMP: Retrieval and Attribute-Marking Enhanced Prompting for Attribute-Controlled Translationcited by 9
  19. 2024On Evaluating Explanation Utility for Human-AI Decision Making in NLPcited by 9
  20. 2024How to Make LLMs Forget: On Reversing In-Context Knowledge Editscited by 9
  21. 2023Quantifying the Plausibility of Context Reliance in Neural Machine Translationcited by 7
  22. 2024Should We Fine-Tune or RAG? Evaluating Different Techniques to Adapt LLMs for Dialoguecited by 7
  23. 2025Bridging the Gap in XAI-Why Reliable Metrics Matter for Explainability and Compliancecited by 7
  24. 2025Source Attribution in Retrieval-Augmented Generationcited by 7
  25. 2023Response Generation in Longitudinal Dialogues: Which Knowledge Representation Helps?cited by 6
  26. 2024Interactive Prompt Debugging with Sequence Saliencecited by 6
  27. 2024SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generationcited by 6
  28. 2025FitCF: A Framework for Automatic Feature Importance-guided Counterfactual Example Generationcited by 6
  29. 2024LLM Attributor: Interactive Visual Attribution for LLM Generationcited by 5
  30. 2024DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learningcited by 5
  31. 2024Attention with Dependency Parsing Augmentation for Fine-Grained Attributioncited by 5
  32. 2025Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translationcited by 5
  33. 2023Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialoguecited by 4
  34. 2024Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfittingcited by 4
  35. 2025Noiser: Bounded Input Perturbations for Attributing Large Language Modelscited by 4
  36. 2025Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreementcited by 4
  37. 2025Hallucinations in Code Change to Natural Language Generation: Prevalence and Evaluation of Detection Metricscited by 4
  38. 2024Variationist: Exploring Multifaceted Variation and Bias in Written Language Datacited by 3

50 most cited of 78 All 78 (JSON)