mlsanityData to 7 Oct 2026

Empowering Language Understanding with Counterfactual Reasoning

arXiv 2106.03046 ↗First posted 6 Jun 2021cs.CL cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Present language understanding methods have demonstrated extraordinary ability of recognizing patterns in texts via machine learning. However, existing methods indiscriminately use the recognized patterns in the testing phase that is inherently different from us humans who have counterfactual thinking, e.g., to scrutinize for the hard testing samples. Inspired by this, we propose a Counterfactual Reasoning Model, which mimics the counterfactual thinking by learning from few counterfactual samples. In particular, we devise a generation module to generate representative counterfactual samples for each factual sample, and a retrospective module to retrospect the model prediction by comparing the counterfactual and factual samples. Extensive experiments on sentiment analysis (SA) and natural language inference (NLI) validate the effectiveness of our method.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 25 of its 43 references that are in the map. First posted on arXiv 6 Jun 2021.

Builds on

25 of 43 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  8. 2018Hierarchical Graph Representation Learning with Differentiable Poolingcited by 786
  9. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  10. 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
  11. 2020Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effectcited by 256
  12. 2019A Unified MRC Framework for Named Entity Recognitioncited by 199
13 more
  1. 2019Addressing Failure Prediction by Learning Model Confidencecited by 183
  2. 2020Counterfactual Samples Synthesizing for Robust Visual Question Answeringcited by 153
  3. 2020An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Modelscited by 133
  4. 2021Counterfactual Generative Networkscited by 111
  5. 2017Learning Independent Causal Mechanismscited by 104
  6. 2019Entity-Relation Extraction as Multi-Turn Question Answeringcited by 96
  7. 2021Counterfactual Zero-Shot and Open-Set Visual Recognitioncited by 95
  8. 2019Coreference Resolution as Query-based Span Predictioncited by 91
  9. 2020Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performancecited by 85
  10. 2020Learning What Makes a Difference from Counterfactual Examples and Gradient Supervisioncited by 66
  11. 2020Generating Plausible Counterfactual Explanations for Deep Transformers in Financial Text Classificationcited by 40
  12. 2020Should Graph Convolution Trust Neighbors? A Simple Causal Inference Methodcited by 25
  13. 2020Counterfactual Variable Control for Robust and Interpretable Question Answeringcited by 4

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 4; 2023: 3; 2024: 1; 2025: 1.

13 papers.

Sort:

By area:

  1. 2022A Review on Language Models as Knowledge Basescited by 156
  2. 2022A Rationale-Centric Framework for Human-in-the-loop Machine Learningcited by 27
  3. 2023Causal Reasoning of Entities and Events in Procedural Textscited by 23
  4. 2021Uncovering Main Causalities for Long-tailed Information Extractioncited by 21
  5. 2021Boosting the Transferability of Video Adversarial Examples via Temporal Translationcited by 21
  6. 2022Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysiscited by 18
  7. 2022Causal Intervention Improves Implicit Sentiment Analysiscited by 16
  8. 2021Causal Incremental Graph Convolution for Recommender System Retrainingcited by 14
  9. 2023Causal Intervention for Abstractive Related Work Generationcited by 5
  10. 2024The Odyssey of Commonsense Causality: From Foundational Benchmarks to Cutting-Edge Reasoningcited by 5
  11. 2025On the Eligibility of LLMs for Counterfactual Reasoning: A Decompositional Studycited by 5
  12. 2021Learning Robust Recommender from Noisy Implicit Feedbackcited by 1
1 more
  1. 2023Do Not Harm Protected Groups in Debiasing Language Representation Modelscited by 0