# Zero-shot LLM-guided Counterfactual Generation: A Case Study on NLP Model Evaluation

arXiv: https://arxiv.org/abs/2405.04793 · HTML: https://mlsanity.com/paper/2405.04793/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 8 May 2024.

## Builds on (most cited first, 22 of 22)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [BAE: BERT-based Adversarial Examples for Text Classification](https://mlsanity.com/paper/2004.01970/) (2020)
- [Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models](https://mlsanity.com/paper/2101.00288/) (2021)
- [A Diagnostic Study of Explainability Techniques for Text Classification](https://mlsanity.com/paper/2009.13295/) (2020)
- [Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text](https://mlsanity.com/paper/2012.04698/) (2020)
- [Towards LLM-guided Causal Explainability for Black-box Text Classifiers](https://mlsanity.com/paper/2309.13340/) (2023)
- [Prompting Large Language Models for Counterfactual Generation: An Empirical Study](https://mlsanity.com/paper/2305.14791/) (2023)
- [Plug and Play Counterfactual Text Generation for Model Robustness](https://mlsanity.com/paper/2206.10429/) (2022)

## Built on by (most cited first, 18 of 18)

- [RATE: Causal Explainability of Reward Models with Imperfect Counterfactuals](https://mlsanity.com/paper/2410.11348/) (2024)
- [FitCF: A Framework for Automatic Feature Importance-guided Counterfactual Example Generation](https://mlsanity.com/paper/2501.00777/) (2025)
- [Truth or Twist? Optimal Model Selection for Reliable Label Flipping Evaluation in LLM-based Counterfactuals](https://mlsanity.com/paper/2505.13972/) (2025)
- [Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanations](https://mlsanity.com/paper/2601.00282/) (2026)
- [Few-Shot Knowledge Distillation of LLMs With Counterfactual Explanations](https://mlsanity.com/paper/2510.21631/) (2025)
- [Counterfactual Modeling with Fine-Tuned LLMs for Health Intervention Design and Sensor Data Augmentation](https://mlsanity.com/paper/2601.14590/) (2026)
- [Show Me How: Benefits and Challenges of Agent-Augmented Counterfactual Explanations for Non-Expert Users](https://mlsanity.com/paper/2504.13897/) (2025)
- [Teach a Reward Model to Correct Itself: Reward Guided Adversarial Failure Discovery for Robust Reward Modeling](https://mlsanity.com/paper/2507.06419/) (2025)
- [Why Are You Wrong? Counterfactual Explanations for Language Grounding with 3D Objects](https://mlsanity.com/paper/2505.06030/) (2025)
- [SenseCF: LLM-Prompted Counterfactuals for Intervention and Sensor Data Augmentation](https://mlsanity.com/paper/2507.05541/) (2025)
- [Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation](https://mlsanity.com/paper/2601.00263/) (2026)
- [iFlip: Iterative Feedback-driven Counterfactual Example Refinement](https://mlsanity.com/paper/2601.01446/) (2026)
- [When Verification Fails: How Compositionally Infeasible Claims Escape Rejection](https://mlsanity.com/paper/2604.10990/) (2026)
- [Agentic Search for Counterfactual Recourse under Fixed LLM Budgets](https://mlsanity.com/paper/2606.08696/) (2026)
- [Neuro-Agentic Control: A Deep Learning-based LLM-Powered Agentic AI Framework for Controlling Security Controls](https://mlsanity.com/paper/2607.09076/) (2026)
- [An Empirical Study of Counterfactual Self-Explanations in LLMs](https://mlsanity.com/paper/2609.17119/) (2026)
- [Counterfactual Self-Evolving Agents for Evidence-Grounded Reasoning](https://mlsanity.com/paper/2609.32870/) (2026)
- [COFFEQA: Counterfactual-Guided Few-Shot Distillation for LLM Question Answering](https://mlsanity.com/paper/2610.12837/) (2026)

## Abstract (arXiv)

With the development and proliferation of large, complex, black-box models for solving many natural language processing (NLP) tasks, there is also an increasing necessity of methods to stress-test these models and provide some degree of interpretability or explainability. While counterfactual examples are useful in this regard, automated generation of counterfactuals is a data and resource intensive process. such methods depend on models such as pre-trained language models that are then fine-tuned on auxiliary, often task-specific datasets, that may be infeasible to build in practice, especially for new tasks and data domains. Therefore, in this work we explore the possibility of leveraging large language models (LLMs) for zero-shot counterfactual generation in order to stress-test NLP models. We propose a structured pipeline to facilitate this generation, and we hypothesize that the instruction-following and textual understanding capabilities of recent LLMs can be effectively leveraged for generating high quality counterfactuals in a zero-shot manner, without requiring any training or fine-tuning. Through comprehensive experiments on a variety of propreitary and open-source LLMs, along with various downstream tasks in NLP, we explore the efficacy of LLMs as zero-shot counterfactual generators in evaluating and explaining black-box NLP models.