# Guiding LLMs to Generate High-Fidelity and High-Quality Counterfactual Explanations for Text Classification

arXiv: https://arxiv.org/abs/2503.04463 · HTML: https://mlsanity.com/paper/2503.04463/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 32 references that are in the map.
- First posted on arXiv 6 Mar 2025.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [Entity-Based Knowledge Conflicts in Question Answering](https://mlsanity.com/paper/2109.05052/) (2021)
- [Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models](https://mlsanity.com/paper/2101.00288/) (2021)
- [Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidence](https://mlsanity.com/paper/2210.13701/) (2022)
- [Explaining NLP Models via Minimal Contrastive Editing (MiCE)](https://mlsanity.com/paper/2012.13985/) (2020)
- [Towards LLM-guided Causal Explainability for Black-box Text Classifiers](https://mlsanity.com/paper/2309.13340/) (2023)
- [DoCoGen: Domain Counterfactual Generation for Low Resource Domain Adaptation](https://mlsanity.com/paper/2202.12350/) (2022)
- [LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Study](https://mlsanity.com/paper/2405.00722/) (2024)
- [CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation](https://mlsanity.com/paper/2210.04873/) (2022)
- [CREST: A Joint Framework for Rationalization and Counterfactual Text Generation](https://mlsanity.com/paper/2305.17075/) (2023)
- [SPECTRA: Sparse Structured Text Rationalization](https://mlsanity.com/paper/2109.04552/) (2021)
- [Text Counterfactuals via Latent Optimization and Shapley-Guided Search](https://mlsanity.com/paper/2110.11589/) (2021)
- [CEval: A Benchmark for Evaluating Counterfactual Text Generation](https://mlsanity.com/paper/2404.17475/) (2024)
- [Revealing the Parametric Knowledge of Language Models: A Unified Framework for Attribution Methods](https://mlsanity.com/paper/2404.18655/) (2024)
- [AutoCAD: Automatically Generating Counterfactuals for Mitigating Shortcut Learning](https://mlsanity.com/paper/2211.16202/) (2022)

## Built on by (most cited first, 3 of 3)

- [Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation](https://mlsanity.com/paper/2601.00263/) (2026)
- [iFlip: Iterative Feedback-driven Counterfactual Example Refinement](https://mlsanity.com/paper/2601.01446/) (2026)
- [Which Modality Decides? Counterfactual Modality Attribution for Multimodal LLMs](https://mlsanity.com/paper/2608.00076/) (2026)

## Abstract (arXiv)

The need for interpretability in deep learning has driven interest in counterfactual explanations, which identify minimal changes to an instance that change a model's prediction. Current counterfactual (CF) generation methods require task-specific fine-tuning and produce low-quality text. Large Language Models (LLMs), though effective for high-quality text generation, struggle with label-flipping counterfactuals (i.e., counterfactuals that change the prediction) without fine-tuning. We introduce two simple classifier-guided approaches to support counterfactual generation by LLMs, eliminating the need for fine-tuning while preserving the strengths of LLMs. Despite their simplicity, our methods outperform state-of-the-art counterfactual generation methods and are effective across different LLMs, highlighting the benefits of guiding counterfactual generation by LLMs with classifier information. We further show that data augmentation by our generated CFs can improve a classifier's robustness. Our analysis reveals a critical issue in counterfactual generation by LLMs: LLMs rely on parametric knowledge rather than faithfully following the classifier.