mlsanityData to 7 Oct 2026

Counterfactual Token Generation in Large Language Models

arXiv 2409.17027 ↗First posted 25 Sep 2024cs.LG cs.AI cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

"Sure, I am happy to generate a story for you: Captain Lyra stood at the helm of her trusty ship, the Maelstrom's Fury, gazing out at the endless sea. [...] Lyra's eyes welled up with tears as she realized the bitter truth - she had sacrificed everything for fleeting riches, and lost the love of her crew, her family, and herself." Although this story, generated by a large language model, is captivating, one may wonder -- how would the story have unfolded if the model had chosen "Captain Maeve" as the protagonist instead? We cannot know. State-of-the-art large language models are stateless -- they maintain no internal memory or state. Given a prompt, they generate a sequence of tokens as an output using an autoregressive process. As a consequence, they cannot reason about counterfactual alternatives to tokens they have generated in the past. In this work, our goal is to enhance them with this functionality. To this end, we develop a causal model of token generation that builds upon the Gumbel-Max structural causal model. Our model allows any large language model to perform counterfactual token generation at almost no cost in comparison with vanilla token generation, it is embarrassingly simple to implement, and it does not require any fine-tuning nor prompt engineering. We implement our model on Llama 3 8B-Instruct and Ministral-8B-Instruct and conduct a qualitative and a quantitative analysis of counterfactually generated text. We conclude with a demonstrative application of counterfactual token generation for bias detection, unveiling interesting insights about the model of the world constructed by large language models.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 35 of its 59 references that are in the map. First posted on arXiv 25 Sep 2024.

Builds on

35 of 59 references.

References · 35

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2023Mistral 7Bcited by 3,065
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2018Sanity Checks for Saliency Mapscited by 1,149
  8. 2017Counterfactual Fairnesscited by 802
  9. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  10. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  11. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 348
  12. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 243
23 more
  1. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  2. 2019Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Modelscited by 114
  3. 2024Evaluating the World Model Implicit in a Generative Modelcited by 107
  4. 2023CLadder: Assessing Causal Reasoning in Language Modelscited by 102
  5. 2019Counterfactual Story Reasoning and Generationcited by 87
  6. 2018Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Searchcited by 81
  7. 2020Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoningcited by 65
  8. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  9. 2023MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Taskscited by 44
  10. 2024Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactualscited by 40
  11. 2021Counterfactual Explanations in Sequential Decision Making Under Uncertaintycited by 36
  12. 2023Prompting Large Language Models for Counterfactual Generation: An Empirical Studycited by 36
  13. 2021CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Modelscited by 33
  14. 2024LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Studycited by 30
  15. 2023The Magic of IF: Investigating Causal Reasoning Abilities in Large Language Models of Codecited by 21
  16. 2024A Survey on Natural Language Counterfactual Generationcited by 21
  17. 2021Counterfactual Temporal Point Processescited by 20
  18. 2021Learning Generalized Gumbel-max Causal Mechanismscited by 16
  19. 2024CEval: A Benchmark for Evaluating Counterfactual Text Generationcited by 15
  20. 2022Counterfactual Inference of Second Opinionscited by 14
  21. 2021Sketch and Customize: A Counterfactual Story Generatorcited by 9
  22. 2022Counterfactual Analysis in Dynamic Latent State Modelscited by 9
  23. 2021Unsupervised Editing for Counterfactual Storiescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 6.

14 papers.

Sort:

By area:

  1. 2025CounterBench: Evaluating and Improving Counterfactual Reasoning in Large Language Modelscited by 19
  2. 2024Gumbel Counterfactual Generation From Language Modelscited by 13
  3. 2025Is Your LLM Overcharging You? Tokenization, Transparency, and Incentivescited by 13
  4. 2025Evaluation of Large Language Models via Coupled Token Generationcited by 7
  5. 2025Auditing Pay-Per-Token in Large Language Modelscited by 4
  6. 2026Compared to What? Baselines and Metrics for Counterfactual Promptingcited by 4
  7. 2025Abstract Counterfactuals for Language Model Agentscited by 2
  8. 2025Large Language Models as Nondeterministic Causal Modelscited by 2
  9. 2026LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactualscited by 1
  10. 2026Re-feeding Is Not Replaying: Measuring Replay Noise in Counterfactual Token-Credit Estimationcited by 1
  11. 2025Vision-Based Mistake Analysis in Procedural Activities: A Review of Advances and Challengescited by 0
  12. 2026Should I Have Expressed a Different Intent? Counterfactual Generation for LLM-Based Autonomous Controlcited by 0
2 more
  1. 2026Better Think Thrice: Learning to Reason Causally with Double Counterfactual Consistencycited by 0
  2. 2026What Process Evaluation of Coding Agents Actually Measures: Action, Task, and Step Are Three Different Levelscited by 0