mlsanityData to 14 Oct 2026

FitCF: A Framework for Automatic Feature Importance-guided Counterfactual Example Generation

arXiv 2501.00777 ↗First posted 1 Jan 2025cs.CL cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Counterfactual examples are widely used in natural language processing (NLP) as valuable data to improve models, and in explainable artificial intelligence (XAI) to understand model behavior. The automated generation of counterfactual examples remains a challenging task even for large language models (LLMs), despite their impressive performance on many tasks. In this paper, we first introduce ZeroCF, a faithful approach for leveraging important words derived from feature attribution methods to generate counterfactual examples in a zero-shot setting. Second, we present a new framework, FitCF, which further verifies aforementioned counterfactuals by label flip verification and then inserts them as demonstrations for few-shot prompting, outperforming two state-of-the-art baselines. Through ablation studies, we identify the importance of each of FitCF's core components in improving the quality of counterfactuals, as assessed through flip rate, perplexity, and similarity measures. Furthermore, we show the effectiveness of LIME and Integrated Gradients as backbone attribution methods for FitCF and find that the number of demonstrations has the largest effect on performance. Finally, we reveal a strong correlation between the faithfulness of feature attribution scores and the quality of generated counterfactuals, which we hope will serve as an important finding for future research in this direction.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 39 of its 55 references that are in the map. First posted on arXiv 1 Jan 2025.

Builds on

39 of 55 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  8. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  9. 2018Hierarchical Neural Story Generationcited by 1,193
  10. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  11. 2019Attention is not Explanationcited by 929
  12. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
27 more
  1. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 530
  2. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 439
  3. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  4. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 222
  5. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 156
  6. 2021Feature Importance-aware Transferable Adversarial Attackscited by 127
  7. 2020Explaining NLP Models via Minimal Contrastive Editing (MiCE)cited by 106
  8. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  9. 2020Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Textcited by 75
  10. 2023A Survey of Large Language Models Attributioncited by 71
  11. 2023How Do In-Context Examples Affect Compositional Generalization?cited by 51
  12. 2023Post Hoc Explanations of Language Models Can Improve Language Modelscited by 50
  13. 2023Towards LLM-guided Causal Explainability for Black-box Text Classifierscited by 44
  14. 2023Prompting Large Language Models for Counterfactual Generation: An Empirical Studycited by 37
  15. 2024LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Studycited by 31
  16. 2022ferret: a Framework for Benchmarking Explainers on Transformerscited by 26
  17. 2023CREST: A Joint Framework for Rationalization and Counterfactual Text Generationcited by 24
  18. 2024Zero-shot LLM-guided Counterfactual Generation: A Case Study on NLP Model Evaluationcited by 18
  19. 2023CATfOOD: Counterfactual Augmented Training for Improving Out-of-Domain Performance and Calibrationcited by 17
  20. 2024LLMCheckup: Conversational Examination of Large Language Models via Interpretability Tools and Self-Explanationscited by 16
  21. 2021Counterfactual Evaluation for Explainable AIcited by 15
  22. 2021Feature Importance Guided Attack: A Model Agnostic Adversarial Attackcited by 10
  23. 2023TIGTEC : Token Importance Guided TExt Counterfactualscited by 8
  24. 2024Towards Unifying Evaluation of Counterfactual Explanations: Leveraging Large Language Models for Human-Centric Assessmentscited by 8
  25. 2024CoXQL: A Dataset for Parsing Explanation Requests in Conversational XAI Systemscited by 5
  26. 2023Investigating the Encoding of Words in BERT's Neurons using Feature Textualizationcited by 3
  27. 2024Does It Make Sense to Explain a Black Box With Another Black Box?cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2026Do LLM Self-Explanations Help Users Predict Model Behavior? Evaluating Counterfactual Simulatability with Pragmatic Perturbationscited by 7
  2. 2025Truth or Twist? Optimal Model Selection for Reliable Label Flipping Evaluation in LLM-based Counterfactualscited by 5
  3. 2026Can Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanationscited by 5
  4. 2026Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generationcited by 1
  5. 2026iFlip: Iterative Feedback-driven Counterfactual Example Refinementcited by 1
  6. 2026Gumbel Machine: Counterfactual Student Writing Generation via Gumbel Noise Steeringcited by 0