mlsanityData to 7 Oct 2026

A causal framework for explaining the predictions of black-box sequence-to-sequence models

arXiv 1707.01943 ↗First posted 6 Jul 2017cs.LGcited by 58

Not on the map. Why?

Abstract (arXiv)

We interpret the predictions of any black-box structured input-structured output model around a specific input-output pair. Our method returns an "explanation" consisting of groups of input-output tokens that are causally related. These dependencies are inferred by querying the black-box model with perturbed inputs, generating a graph over tokens from the responses, and solving a partitioning problem to select the most relevant components. We focus the general approach on sequence-to-sequence problems, adopting a variational autoencoder to yield meaningful input perturbations. We test our method across several NLP sequence generation tasks.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 20; 2022: 10; 2023: 19; 2024: 4; 2025: 5.

58 papers.

Sort:

  1. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  2. 2022Discovering Invariant Rationales for Graph Neural Networkscited by 186
  3. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  4. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  5. 2021Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Casescited by 77
  6. 2023Universal Information Extraction as Unified Semantic Matchingcited by 47
  7. 2022VALUE: Understanding Dialect Disparity in NLUcited by 45
  8. 2023Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspectioncited by 44
  9. 2022On the Explainability of Natural Language Processing Deep Modelscited by 37
  10. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  11. 2022Can Rationalization Improve Robustness?cited by 36
  12. 2022Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theorycited by 32
38 more
  1. 2021Local Interpretations for Explainable Natural Language Processing: A Surveycited by 31
  2. 2021Rationales for Sequential Predictionscited by 26
  3. 2024Mechanistic?cited by 25
  4. 2023Rationalization for Explainable NLP: A Surveycited by 22
  5. 2021Rationalization through Conceptscited by 19
  6. 2021From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidencecited by 17
  7. 2021Counterfactual Evaluation for Explainable AIcited by 15
  8. 2022A Fine-grained Interpretability Evaluation Benchmark for Neural NLPcited by 15
  9. 2021Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysiscited by 14
  10. 2023A Review of the Role of Causality in Developing Trustworthy AI Systemscited by 14
  11. 2024Towards Safety and Helpfulness Balanced Responses via Controllable Large Language Modelscited by 14
  12. 2021Entropic Causal Inference: Identifiability and Finite Sample Resultscited by 12
  13. 2022On the Probability of Necessity and Sufficiency of Explaining Graph Neural Networks: A Lower Bound Optimization Approachcited by 11
  14. 2021Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?cited by 10
  15. 2021Matching Learned Causal Effects of Neural Networks with Domain Priorscited by 10
  16. 2021Let the CAT out of the bag: Contrastive Attributed explanations for Textcited by 9
  17. 2022Influence Functions for Sequence Tagging Modelscited by 9
  18. 2023Measuring Faithful and Plausible Visual Grounding in VQAcited by 8
  19. 2023Explicit Syntactic Guidance for Neural Text Generationcited by 8
  20. 2021Local Explanation of Dialogue Response Generationcited by 7
  21. 2023Explaining Language Models' Predictions with High-Impact Conceptscited by 7
  22. 2023On the Interplay between Fairness and Explainabilitycited by 7
  23. 2021What classifiers know what they don't?cited by 6
  24. 2023Tackling Interpretability in Audio Classification Networks with Non-negative Matrix Factorizationcited by 5
  25. 2023May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainabilitycited by 5
  26. 2025Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Termscited by 5
  27. 2022An Interpretability Evaluation Benchmark for Pre-trained Language Modelscited by 4
  28. 2025Fake News Detection After LLM Laundering: Measurement and Explanationcited by 4
  29. 2021Explaining Natural Language Processing Classifiers with Occlusion and Language Modelingcited by 2
  30. 2023Towards Learning and Explaining Indirect Causal Effects in Neural Networkscited by 2
  31. 2023Explainable AI for clinical risk prediction: a survey of concepts, methods, and modalitiescited by 2
  32. 2023Using Natural Language Explanations to Improve Robustness of In-context Learningcited by 2
  33. 2021Hierarchical Variational Autoencoder for Visual Counterfactualscited by 1
  34. 2021Explaining the Deep Natural Language Processing by Mining Textual Interpretable Featurescited by 1
  35. 2022Leveraging Causal Inference for Explainable Automatic Program Repaircited by 1
  36. 2023NxPlain: Web-based Tool for Discovery of Latent Conceptscited by 1
  37. 2024Transparent Neighborhood Approximation for Text Classifier Explanationcited by 1
  38. 2025Conversational Explanations: Discussing Explainable AI with Non-AI Expertscited by 1

50 most cited of 58 All 58 (JSON)