mlsanityData to 14 Oct 2026

Shaking the foundations: delusions in sequence models for interaction and control

arXiv 2110.10819 ↗First posted 20 Oct 2021cs.LG cs.AIcited by 62

Not on the map. Why?

Abstract (arXiv)

The recent phenomenal success of language models has reinvigorated machine learning research, and large sequence models such as transformers are being applied to a variety of domains. One important problem class that has remained relatively elusive however is purposeful adaptive behavior. Currently there is a common perception that sequence models "lack the understanding of the cause and effect of their actions" leading them to draw incorrect inferences due to auto-suggestive delusions. In this report we explain where this mismatch originates, and show that it can be resolved by treating actions as causal interventions. Finally, we show that in supervised learning, one can teach a system to condition or intervene on data by training with factual and counterfactual error signals respectively.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Builds on 13 of its 27 references that are in the map. First posted on arXiv 20 Oct 2021.

Builds on

13 of 27 references.

References · 13

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  3. 2019Invariant Risk Minimizationcited by 1,854
  4. 2016Generative Adversarial Imitation Learningcited by 1,597
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  6. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  7. 2016Learning to reinforcement learncited by 377
  8. 2016Why is Posterior Sampling Better than Optimism for Reinforcement Learning?cited by 117
  9. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  10. 2020Causally Correct Partial Models for Reinforcement Learningcited by 24
  11. 2016Thompson Sampling is Asymptotically Optimal in General Environmentscited by 8
  12. 2014Compress and Controlcited by 7
1 more
  1. 2021Causal Analysis of Agent Behavior for AI Safetycited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 17; 2023: 17; 2024: 14; 2025: 6; 2026: 6.

62 papers.

Sort:

By area:

  1. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  2. 2022A Generalist Agentcited by 808
  3. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  4. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  5. 2023Language Modeling Is Compressioncited by 225
  6. 2022Multi-Game Decision Transformerscited by 214
  7. 2022Online Decision Transformercited by 196
  8. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 178
  9. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 156
  10. 2022Language Model Cascadescited by 107
  11. 2022Meaning without reference in large language modelscited by 76
  12. 2023Eight Things to Know about Large Language Modelscited by 73
38 more
  1. 2022You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environmentscited by 63
  2. 2023SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Modelscited by 57
  3. 2023Understanding Causality with Large Language Models: Feasibility and Opportunitiescited by 53
  4. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 44
  5. 2021Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learningcited by 40
  6. 2021CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Modelscited by 34
  7. 2022Sequence Model Imitation Learning with Unobserved Contextscited by 34
  8. 2023A Survey on Knowledge Editing of Neural Networkscited by 34
  9. 2024Better than Your Teacher: LLM Agents that learn from Privileged AI Feedbackcited by 24
  10. 2023Passive learning of active causal strategies in agents and language modelscited by 21
  11. 2024Retrieval-Augmented Decision Transformer: External Memory for In-context RLcited by 21
  12. 2022General Intelligence Requires Rethinking Explorationcited by 20
  13. 2024A social path to human-like artificial intelligencecited by 20
  14. 2024Experts Don't Cheat: Learning What You Don't Know By Predicting Pairscited by 19
  15. 2022Fighting Fire with Fire: Avoiding DNN Shortcuts through Primingcited by 15
  16. 2023Causal Semantic Communication for Digital Twins: A Generalizable Imitation Learning Approachcited by 14
  17. 2023CAR-DESPOT: Causally-Informed Online POMDP Planning for Robots in Confounded Environmentscited by 12
  18. 2023Emergent Causality and the Foundation of Consciousnesscited by 9
  19. 2023Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertaintycited by 9
  20. 2024Position: Foundation Agents as the Paradigm Shift for Decision Makingcited by 9
  21. 2024Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Drivingcited by 8
  22. 2023Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confoundingcited by 7
  23. 2025Advancing the Scientific Method with Large Language Models: From Hypothesis to Discoverycited by 7
  24. 2025Man-Made Heuristics Are Dead. Long Live Code Generators!cited by 6
  25. 2022Are All Vision Models Created Equal? A Study of the Open-Loop to Closed-Loop Causality Gapcited by 5
  26. 2022Deconfounding Imitation Learning with Variational Inferencecited by 5
  27. 2023On the Unexpected Abilities of Large Language Modelscited by 5
  28. 2023Initial State Interventions for Deconfounded Imitation Learningcited by 4
  29. 2023FoMo Rewards: Can we cast foundation models as reward functions?cited by 4
  30. 2024Adversarially Robust Decision Transformercited by 4
  31. 2026From AGI to ASIcited by 4
  32. 2024IGDrivSim: A Benchmark for the Imitation Gap in Autonomous Drivingcited by 3
  33. 2022DNNFuser: Generative Pre-Trained Transformer as a Generalized Mapper for Layer Fusion in DNN Acceleratorscited by 2
  34. 2022Uniqueness and Complexity of Inverse MDP Modelscited by 2
  35. 2023Predictive Minds: LLMs As Atypical Active Inference Agentscited by 2
  36. 2024A Bayesian Solution To The Imitation Gapcited by 2
  37. 2024Unsupervised Cognitioncited by 2
  38. 2025Learning to Decide with Just Enough: Information-Theoretic Context Summarization for CMDPscited by 2

50 most cited of 62 All 62 (JSON)