mlsanityData to 14 Oct 2026

Composition Collapse: Stable Factual Knowledge Does Not Imply Compositional Reasoning

arXiv 2605.26789 ↗First posted 26 May 2026cs.AIcited by 1

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Post-training is routinely evaluated through aggregate benchmark scores that treat multi-hop reasoning as a single capability -- as if a model that answers more questions correctly must be better at assembling facts. We show that this assumption can be misleading: recipes with statistically indistinguishable atomic knowledge produce composition behaviour separated by over 40 percentage points, a phenomenon we call composition collapse: the systematic failure to assemble stably-known facts into chains, invisible to aggregate metrics. We introduce a double-gate protocol that changes the estimand from an aggregate compositionality gap to residual composition failure conditioned on stable atomic access, decomposing post-training gains into three independent channels: atomic stability, residual composition, and critical depth. On a benchmark of temporal factual chains spanning depths 2--11 across four post-training recipes, this decomposition reveals that post-training objectives shift composition capability in directions that aggregate metrics mask, and suggests that claims about multi-hop reasoning improvement should be accompanied by atomic-gate-controlled composition metrics. Diagnostic probes further show that a substantial share of measured composition failure reflects generation-time computation constraints rather than permanent inability to compose.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 9 of its 19 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

9 of 19 references.

References · 9

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  6. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  7. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 270
  8. 2024Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queriescited by 78
  9. 2024Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?cited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Multi-Hop Knowledge Composition is Bound by Pretraining Exposure
  2. 2026Wrong Prediction, Right Answer: Recovering Evidence from Collapsed LLM Sequence Scores
  3. 2026The Ignition Is Real, and It Lives at the Readout: Latent composition, difficulty-clocked ignition, and the interface-constituted commit in a recurrent-depth reasoner
  4. 2024Understanding and Patching Compositional Reasoning in LLMs
  5. 2026Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation
  6. 2026Track, Rank, Crack: Epistemic Working Memory Scales Multi-Hop Reasoning in Language Agents
  7. 2025Learning Composable Chains-of-Thought
  8. 2025Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning