mlsanityData to 7 Oct 2026

Distribution Matching for Rationalization

arXiv 2106.00320 ↗First posted 1 Jun 2021cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

The task of rationalization aims to extract pieces of input text as rationales to justify neural network predictions on text classification tasks. By definition, rationales represent key text pieces used for prediction and thus should have similar classification feature distribution compared to the original input text. However, previous methods mainly focused on maximizing the mutual information between rationales and labels while neglecting the relationship between rationales and input text. To address this issue, we propose a novel rationalization method that matches the distributions of rationales and input text in both the feature space and output space. Empirically, the proposed distribution matching approach consistently outperforms previous methods by a large margin. Our data and code are available.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 27 of its 34 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

27 of 34 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2018Deep contextualized word representationscited by 2,285
  8. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  9. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  10. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  11. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,097
15 more
  1. 2018Born Again Neural Networkscited by 539
  2. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  3. 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 312
  4. 2016Rationalizing Neural Predictionscited by 281
  5. 2015Generative Moment Matching Networkscited by 252
  6. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  7. 2017Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learningcited by 192
  8. 2020Invariant Rationalizationcited by 131
  9. 2017Inferring and Executing Programs for Visual Reasoningcited by 130
  10. 2019Interpretable Neural Predictions with Differentiable Binary Variablescited by 126
  11. 2019Rethinking Cooperative Rationalization: Introspective Extraction and Complement Controlcited by 96
  12. 2016Learning to Compose Neural Networks for Question Answeringcited by 85
  13. 2020An Information Bottleneck Approach for Controlling Conciseness in Rationale Extractioncited by 69
  14. 2018Deriving Machine Attention from Human Rationalescited by 54
  15. 2019A Game Theoretic Approach to Class-wise Selective Rationalizationcited by 32

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 5; 2024: 3; 2025: 4.

14 papers.

Sort:

  1. 2023D-Separation for Causal Self-Explanationcited by 21
  2. 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
  3. 2022FR: Folded Rationalization with a Unified Encodercited by 18
  4. 2023Enhancing the Rationale-Input Alignment for Self-explaining Rationalizationcited by 15
  5. 2023Decoupled Rationalization with Asymmetric Learning Rates: A Flexible Lipschitz Restraintcited by 14
  6. 2023MGR: Multi-generator Based Rationalizationcited by 12
  7. 2024Towards Faithful Explanations: Boosting Rationalization with Shortcuts Discoverycited by 8
  8. 2025Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilizationcited by 6
  9. 2025Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasetscited by 6
  10. 2024MARE: Multi-Aspect Rationale Extractor on Unsupervised Rationale Extractioncited by 4
  11. 2023You Only Forward Once: Prediction and Rationalization in A Single Forward Passcited by 2
  12. 2025Boosting Explainability through Selective Rationalization in Pre-trained Language Modelscited by 2
2 more
  1. 2025Learnable Game-theoretic Policy Optimization for Data-centric Self-explanation Rationalizationcited by 1
  2. 2024Interlocking-free Selective Rationalization Through Genetic-based Learningcited by 0