mlsanityData to 14 Oct 2026

Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics

arXiv 2405.04669 ↗First posted 7 May 2024cs.LG cs.CLcited by 27

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Auto-regressive large language models (LLMs) show impressive capacities to solve many complex reasoning tasks while struggling with some simple logical reasoning tasks such as inverse search: when trained on 'A → B' (e.g., 'Tom is the parent of John'), LLM fails to directly conclude 'B gets A' (e.g., 'John is the child of Tom') during inference even if the two sentences are semantically identical, which is known as the 'reversal curse'. In this paper, we theoretically analyze the reversal curse via the training dynamics of (stochastic) gradient descent for two auto-regressive models: (1) a bilinear model that can be viewed as a simplification of a one-layer transformer; (2) one-layer transformers under certain assumptions. Our analysis reveals that for both models, the reversal curse is a consequence of the (effective) model weights 'asymmetry', i.e., the increase of weights from a token A to token B during training does not necessarily cause the increase of the weights from B to A, which is caused by the training dynamics under certain choice of loss function and the optimization space of model parameters. Moreover, our analysis can be naturally applied to other logical reasoning tasks such as chain-of-thought (COT), which provides a new perspective different from previous work that focuses on expressivity. Finally, we conduct experiments to validate our theory on multi-layer transformers under different settings. Our code is available at https://github.com/marlo-z/reversalcurseanalysis/.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 58 of its 68 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

58 of 68 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022STaR: Bootstrapping Reasoning With Reasoning28 Mar

Other references · 57

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  9. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  10. 2022In-context Learning and Induction Headscited by 964
  11. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  12. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
38 more
  1. 2022Transformers learn in-context by gradient descentcited by 732
  2. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  3. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
  4. 2018Universal Transformerscited by 647
  5. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  6. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  7. 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 406
  8. 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 341
  9. 2023Trained Transformers Learn Linear Models In-Contextcited by 337
  10. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 298
  11. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 294
  12. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 288
  13. 2022Transformers Learn Shortcuts to Automatacited by 268
  14. 2022Exploring Length Generalization in Large Language Modelscited by 206
  15. 2022Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limitcited by 187
  16. 2023One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attentioncited by 170
  17. 2023Birth of a Transformer: A Memory Viewpointcited by 161
  18. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  19. 2021Inductive Biases and Variable Creation in Self-Attention Mechanismscited by 144
  20. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 143
  21. 2020Infinite attention: NNGP and NTK for deep attention networkscited by 121
  22. 2024How Transformers Learn Causal Structure with Gradient Descentcited by 118
  23. 2023In-Context Convergence of Transformerscited by 115
  24. 2021Self-Attention Networks Can Process Bounded Hierarchical Languagescited by 105
  25. 2021Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformerscited by 102
  26. 2023Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformercited by 102
  27. 2022Vision Transformers provably learn spatial structurecited by 93
  28. 2020On the Computational Power of Transformers and its Implications in Sequence Modelingcited by 85
  29. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 83
  30. 2023Transformers learn through gradual rank increasecited by 52
  31. 2023JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attentioncited by 52
  32. 2024A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Taskcited by 49
  33. 2024Reverse Training to Nurse the Reversal Cursecited by 49
  34. 2023Do Transformers Parse while Predicting the Masked Word?cited by 48
  35. 2021On the Expressive Power of Self-Attention Matricescited by 47
  36. 2023The Closeness of In-Context Learning and Weight Shifting for Softmax Regressioncited by 41
  37. 2023What can a Single Attention Layer Learn? A Study Through the Random Features Lenscited by 40
  38. 2023Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Modelscited by 39

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 14; 2026: 9.

27 papers.

Sort:

By area:

  1. 2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoningcited by 91
  2. 2024Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMscited by 56
  3. 2024Transformers Provably Solve Parity Efficiently with Chain of Thoughtcited by 50
  4. 2026Large Language Model Reasoning Failurescited by 21
  5. 2025How Do LLMs Perform Two-Hop Reasoning in Context?cited by 20
  6. 2025Is the Reversal Curse a Binding Problem? Uncovering Limitations of Transformers from a Basic Generalization Failurecited by 13
  7. 2024Delving into the Reversal Curse: How Far Can Large Language Models Generalize?cited by 11
  8. 2025Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoningcited by 10
  9. 2025Generalization or Hallucination? Understanding Out-of-Context Reasoning in Transformerscited by 10
  10. 2025Benefits and Pitfalls of Reinforcement Learning for Language Model Planning: A Theoretical Perspectivecited by 4
  11. 2025Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Modelscited by 3
  12. 2024Reverse Modeling in Large Language Modelscited by 2
15 more
  1. 2025Review of Hallucination Understanding in Large Language and Vision Modelscited by 2
  2. 2025Diffusion-Inspired Masked Fine-Tuning for Knowledge Injection in Autoregressive LLMscited by 2
  3. 2026Breaking the Reversal Curse in Autoregressive Language Models via Identity Bridgecited by 2
  4. 2026Transformers Provably Learn to Internalize Chain-of-Thoughtcited by 2
  5. 2025Open Problems and a Hypothetical Path Forward in LLM Knowledge Paradigmscited by 1
  6. 2025Bilinear representation mitigates reversal curse and enables consistent model editingcited by 1
  7. 2026DiffER: Diffusion Entity-Relation Modeling for Reversal Curse in Diffusion Large Language Modelscited by 1
  8. 2025Multimodal Integrated Knowledge Transfer to Large Language Models through Preference Optimization with Biomedical Applicationscited by 0
  9. 2025VLMs Can Aggregate Scattered Training Patchescited by 0
  10. 2025GSM-Agent: Understanding Agentic Reasoning Using Controllable Environmentscited by 0
  11. 2026Memorization, Emergence, and Explaining Reversal Failures: A Controlled Study of Relational Semantics in LLMscited by 0
  12. 2026A Theoretical Analysis of Why Masked Diffusion Models Mitigate the Reversal Cursecited by 0
  13. 2026Feature Resemblance: Towards a Theoretical Understanding of Analogical Reasoning in Transformerscited by 0
  14. 2026Multi-Hop Knowledge Composition is Bound by Pretraining Exposurecited by 0
  15. 2026Distilling Directional Verificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Illusion of Latent Generalization: Bi-directionality and the Reversal Curse
  2. 2023Exploring the Reversal Curse and Other Deductive Logical Reasoning in BERT and GPT-Based Large Language Models
  3. 2025LEDOM: Reverse Language Model
  4. 2024Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up
  5. 2026How Does Unfaithful Reasoning Emerge from Autoregressive Training? A Study of Synthetic Experiments
  6. 2024Reverse Thinking Makes LLMs Stronger Reasoners
  7. 2026A Mechanistic Analysis of Looped Reasoning Language Models
  8. 2026Brevity Constraints Reverse Performance Hierarchies in Language Models