mlsanityData to 7 Oct 2026

Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning

arXiv 2010.05906 ↗First posted 12 Oct 2020cs.CL cs.AI cs.LGcited by 65

Not on the map. Why?

Abstract (arXiv)

Abductive and counterfactual reasoning, core abilities of everyday human cognition, require reasoning about what might have happened at time t, while conditioning on multiple contexts from the relative past and future. However, simultaneous incorporation of past and future contexts using generative language models (LMs) can be challenging, as they are trained either to condition only on the past context or to perform narrowly scoped text-infilling. In this paper, we propose DeLorean, a new unsupervised decoding algorithm that can flexibly incorporate both the past and future contexts using only off-the-shelf, left-to-right language models and no supervision. The key intuition of our algorithm is incorporating the future through back-propagation, during which, we only update the internal representation of the output while fixing the model parameters. By alternating between forward and backward propagation, DeLorean can decode the output representation that reflects both the left and right contexts. We demonstrate that our approach is general and applicable to two nonmonotonic reasoning tasks: abductive text generation and counterfactual story revision, where DeLorean outperforms a range of unsupervised and some supervised methods, based on automatic and human evaluation.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 19; 2023: 12; 2024: 12; 2025: 6; 2026: 6.

65 papers.

Sort:

By area:

  1. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  2. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  3. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 243
  4. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  5. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 159
  6. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 96
  7. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  8. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  9. 2021Probing Across Time: What Does RoBERTa Know and When?cited by 73
  10. 2021Controlled Text Generation as Continuous Optimization with Multiple Constraintscited by 70
  11. 2022Gradient-Based Constrained Sampling from Language Modelscited by 64
  12. 2023Natural Language Reasoning, A Surveycited by 61
38 more
  1. 2022ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classificationcited by 45
  2. 2023KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detectioncited by 43
  3. 2022Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Surveycited by 38
  4. 2022The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoningcited by 38
  5. 2023Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuningcited by 33
  6. 2024DeepEdit: Knowledge Editing as Decoding with Constraintscited by 33
  7. 2022Composable Text Controls in Latent Space with ODEscited by 28
  8. 2021GENIE: Toward Reproducible and Standardized Human Evaluation for Text Generationcited by 20
  9. 2022On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlationscited by 19
  10. 2021Unsupervised Enrichment of Persona-grounded Dialog with Background Storiescited by 14
  11. 2024Counterfactual Token Generation in Large Language Modelscited by 14
  12. 2021COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completioncited by 12
  13. 2022Knowledge Infused Decodingcited by 12
  14. 2025LogiDynamics: Unraveling the Dynamics of Inductive, Abductive and Deductive Logical Inferences in LLM Reasoningcited by 12
  15. 2021Element Intervention for Open Relation Extractioncited by 11
  16. 2023Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanationscited by 11
  17. 2025Controllable Logical Hypothesis Generation for Abductive Reasoning in Knowledge Graphscited by 11
  18. 2021Transferable Persona-Grounded Dialogues via Grounded Minimal Editscited by 10
  19. 2022Coalescing Global and Local Information for Procedural Text Understandingcited by 10
  20. 2023Fill in the Blank: Exploring and Enhancing LLM Capabilities for Backward Reasoning in Math Word Problemscited by 9
  21. 2024Black Swan: Abductive and Defeasible Video Reasoning in Unpredictable Eventscited by 9
  22. 2022Target-Guided Dialogue Response Generation Using Commonsense and Data Augmentationcited by 8
  23. 2025Unifying Deductive and Abductive Reasoning in Knowledge Graphs with Masked Diffusion Modelcited by 8
  24. 2023Toward Unified Controllable Text Generation via Regular Expression Instructioncited by 7
  25. 2021Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentationcited by 6
  26. 2023EvEval: A Comprehensive Evaluation of Event Semantics for Large Language Modelscited by 6
  27. 2023Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoderscited by 6
  28. 2024RNR: Teaching Large Language Models to Follow Roles and Rulescited by 6
  29. 2022Counterfactual Recipe Generation: Exploring Compositional Generalization in a Realistic Scenariocited by 5
  30. 2024generAItor: Tree-in-the-Loop Text Generation for Language Model Explainability and Adaptationcited by 5
  31. 2026Wiring the 'Why': A Unified Taxonomy and Survey of Abductive Reasoning in LLMscited by 5
  32. 2022Twist Decoding: Diverse Generators Guide Each Othercited by 4
  33. 2023PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planningcited by 4
  34. 2023Exploring the Potential of Multi-Modal AI for Driving Hazard Predictioncited by 4
  35. 2024Language Rectified Flow: Advancing Diffusion Language Generation with Probabilistic Flowscited by 4
  36. 2024A Comprehensive Evaluation on Event Reasoning of Large Language Modelscited by 4
  37. 2021Reconsidering the Past: Optimizing Hidden States in Language Modelscited by 3
  38. 2022Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injectioncited by 3

50 most cited of 65 All 65 (JSON)