mlsanityData to 7 Oct 2026

Dynamics Reveals Structure: Challenging the Linear Propagation Assumption

arXiv 2601.21601 ↗First posted 29 Jan 2026cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Neural networks adapt through first-order parameter updates, yet it remains unclear whether such updates preserve logical coherence. We investigate the geometric limits of the Linear Propagation Assumption (LPA), the premise that local updates coherently propagate to logical consequences. To formalize this, we adopt relation algebra and study three core operations on relations: negation flips truth values, converse swaps argument order, and composition chains relations. For negation and converse, we prove that guaranteeing direction-agnostic first-order propagation necessitates a tensor factorization separating entity-pair context from relation content. However, for composition, we identify a fundamental obstruction. We show that composition reduces to conjunction, and prove that any conjunction well-defined on linear features must be bilinear. Since bilinearity is incompatible with negation, this forces the feature map to collapse. These results suggest that failures in knowledge editing, the reversal curse, and multi-hop reasoning may stem from common structural limitations inherent to the LPA.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

33 of 47 references.

References · 33

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  8. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  9. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  10. 2016Group Equivariant Convolutional Networkscited by 1,154
  11. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  12. 2022Mass-Editing Memory in a Transformercited by 976
21 more
  1. 2022Toy Models of Superpositioncited by 928
  2. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  3. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  4. 2018On Lazy Training in Differentiable Programmingcited by 671
  5. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  6. 2022Memory-Based Model Editing at Scalecited by 443
  7. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
  8. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  9. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  10. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  11. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 238
  12. 2020On the Binding Problem in Artificial Neural Networkscited by 230
  13. 2020Editable Neural Networkscited by 154
  14. 2020Rewriting a Deep Generative Modelcited by 105
  15. 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 86
  16. 2025Open Problems in Machine Unlearning for AI Safetycited by 49
  17. 2024Fundamental Problems With Model Editing: How Should Rational Belief Revision Work in LLMs?cited by 24
  18. 2025Is the Reversal Curse a Binding Problem? Uncovering Limitations of Transformers from a Basic Generalization Failurecited by 13
  19. 2024Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Modelscited by 8
  20. 2025Is Model Editing Built on Sand? Revealing Its Illusory Success and Fragile Foundationcited by 6
  21. 2025Characterizing Pattern Matching and Its Limits on Compositional Task Structurescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026What are the Right Symmetries for Formal Theorem Proving?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models
  2. 2026Composition Collapse: Stable Factual Knowledge Does Not Imply Compositional Reasoning
  3. 2026Unveiling the Reasoning Process of Large Language Models
  4. 2026The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations
  5. 2026Syntax vs. Semantics: How Transformers Learn Deep Dependencies
  6. 2026Grounding LLM Reasoning under Incomplete Graph Evidence
  7. 2025The Geometry of Reasoning: Flowing Logics in Representation Space
  8. 2026The Geometry of Thought: How Scale Restructures Reasoning In Large Language Models