mlsanityData to 13 Oct 2026

A Survey on Natural Language Counterfactual Generation

arXiv 2407.03993 ↗First posted 4 Jul 2024cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

Natural language counterfactual generation aims to minimally modify a given text such that the modified text will be classified into a different class. The generated counterfactuals provide insight into the reasoning behind a model's predictions by highlighting which words significantly influence the outcomes. Additionally, they can be used to detect model fairness issues and augment the training data to enhance the model's robustness. A substantial amount of research has been conducted to generate counterfactuals for various NLP tasks, employing different models and methodologies. With the rapid growth of studies in this field, a systematic review is crucial to guide future researchers and developers. To bridge this gap, this survey provides a comprehensive overview of textual counterfactual generation methods, particularly those based on Large Language Models. We propose a new taxonomy that systematically categorizes the generation methods into four groups and summarizes the metrics for evaluating the generation quality. Finally, we discuss ongoing research challenges and outline promising directions for future work.

Key facts. Cited by 21 papers in the map (data through 13 Oct 2026). Builds on 74 of its 120 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

74 of 120 references.

References · 74

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2020Language Models are Few-Shot Learnerscited by 32,320
  3. 2023GPT-4 Technical Reportcited by 18,278
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,107
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,515
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,480
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,317
  8. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,055
  9. 2017Axiomatic Attribution for Deep Networkscited by 3,368
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,360
  11. 2020Shortcut Learning in Deep Neural Networkscited by 1,944
  12. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,203
38 more
  1. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,088
  2. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,071
  3. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 992
  4. 2017Counterfactual Fairnesscited by 804
  5. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 797
  6. 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 614
  7. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 490
  8. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 486
  9. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 433
  10. 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 416
  11. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 400
  12. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 352
  13. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  14. 2017Toward Controlled Generation of Textcited by 272
  15. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 232
  16. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  17. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 210
  18. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  19. 2018Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfercited by 191
  20. 2020Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Reviewcited by 174
  21. 2020What shapes feature representations? Exploring datasets, architectures, and trainingcited by 126
  22. 2020Explaining NLP Models via Minimal Contrastive Editing (MiCE)cited by 105
  23. 2019Counterfactual Story Reasoning and Generationcited by 87
  24. 2019Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfercited by 81
  25. 2020Logic-Guided Data Augmentation and Regularization for Consistent Question Answeringcited by 78
  26. 2020Politeness Transfer: A Tag and Generate Approachcited by 78
  27. 2020Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Textcited by 75
  28. 2020Robustness to Spurious Correlations in Text Classification via Automatically Generated Counterfactualscited by 67
  29. 2021Tailor: Generating and Perturbing Text with Semantic Controlscited by 67
  30. 2019"Mask and Infill" : Applying Masked Language Model to Sentiment Transfercited by 48
  31. 2020"Call me sexist, but...": Revisiting Sexism Detection Using Psychological Scales and Adversarial Samplescited by 48
  32. 2023Explore Spurious Correlations at the Concept Level in Language Models for Text Classificationcited by 48
  33. 2020Model extraction from counterfactual explanationscited by 47
  34. 2022A Causal Lens for Controllable Text Generationcited by 45
  35. 2023Towards LLM-guided Causal Explainability for Black-box Text Classifierscited by 44
  36. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  37. 2021An Investigation of the (In)effectiveness of Counterfactually Augmented Datacited by 42
  38. 2020Generating Plausible Counterfactual Explanations for Deep Transformers in Financial Text Classificationcited by 40

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 2; 2025: 10; 2026: 9.

21 papers.

Sort:

By area:

  1. 2025LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanationscited by 16
  2. 2024Counterfactual Token Generation in Large Language Modelscited by 14
  3. 2025Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Frameworkcited by 10
  4. 2025What Has Been Lost with Synthetic Evaluation?cited by 9
  5. 2025On the Eligibility of LLMs for Counterfactual Reasoning: A Decompositional Studycited by 5
  6. 2024Language Models as Causal Effect Generatorscited by 4
  7. 2026All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detectioncited by 4
  8. 2026JevOut: Natural Context Can Flip Decision Modelscited by 4
  9. 2025Can LLMs Simulate Personas with Reversed Performance? A Systematic Investigation for Counterfactual Instruction Following in Math Reasoning Contextcited by 3
  10. 2025Can Hallucinations Help? Boosting LLMs for Drug Discoverycited by 2
  11. 2026LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactualscited by 2
  12. 2025Success is in the Details: Evaluate and Enhance Details Sensitivity of Code LLMs through Counterfactualscited by 1
9 more
  1. 2025Deriving Strategic Market Insights with Large Language Models: A Benchmark for Forward Counterfactual Generationcited by 1
  2. 2025When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMscited by 1
  3. 2026Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generationcited by 1
  4. 2026iFlip: Iterative Feedback-driven Counterfactual Example Refinementcited by 1
  5. 2025CoBA: Counterbias Text Augmentation for Mitigating Various Spurious Correlations via Semantic Triplescited by 0
  6. 2026Learning Half-Spaces from Perturbed Contrastive Examplescited by 0
  7. 2026Gumbel Machine: Counterfactual Student Writing Generation via Gumbel Noise Steeringcited by 0
  8. 2026CoCoGEC: Counterfactual Generation for Robust Grammatical Error Correctioncited by 0
  9. 2026Contextual Causality with Large Language Models: A Surveycited by 0