A Brain-like Synergistic Core in LLMs Drives Behaviour and Learning
Not on the map. Why?
Abstract (arXiv)
The independent evolution of intelligence in biological and artificial systems offers a unique opportunity to identify its fundamental computational principles. Here we show that large language models spontaneously develop synergistic cores -- components where information integration exceeds individual parts -- remarkably similar to those in the human brain. Using principles of information decomposition across multiple LLM model families and architectures, we find that areas in middle layers exhibit synergistic processing while early and late layers rely on redundancy, mirroring the informational organisation in biological brains. This organisation emerges through learning and is absent in randomly initialised networks. Crucially, ablating synergistic components causes disproportionate behavioural changes and performance loss, aligning with theoretical predictions about the fragility of synergy. Moreover, fine-tuning synergistic regions through reinforcement learning yields significantly greater performance gains than training redundant components, yet supervised fine-tuning shows no such advantage. This convergence suggests that synergistic information processing is a fundamental property of intelligence, providing targets for principled model design and testable predictions for biological intelligence.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 29 of its 54 references that are in the map. First posted on arXiv 11 Jan 2026.
Builds on
29 of 54 references.
References · 29
- 2017Attention Is All You Needcited by 60,361
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
- 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
- 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
- 2022Toy Models of Superpositioncited by 971
- 2022In-context Learning and Induction Headscited by 964
- 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
17 more
- 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 383
- 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
- 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 200
- 2024Transformer Layers as Painterscited by 54
- 2018Understanding Convolutional Neural Networks with Information Theory: An Initial Explorationcited by 32
- 2025Intermediate Layer Classifiers for OOD generalizationcited by 14
- 2022A Measure of the Complexity of Neural Representations based on Partial Information Decompositioncited by 11
- 2025Learning to Skip the Middle Layers of Transformerscited by 10
- 2022Multivariate Information Theory Uncovers Synergistic Subsystems of the Human Cerebral Cortexcited by 8
- 2022Synergistic information supports modality integration and flexible learning in neural networks solving multiple taskscited by 8
- 2023Partial entropy decomposition reveals higher-order structures in human brain activitycited by 8
- 2023A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functionscited by 8
- 2025Layerwise Importance Analysis of Feed-Forward Networks in Transformer-based Language Modelscited by 6
- 2021Integrated information as a common signature of dynamical and information-processing complexitycited by 5
- 2021Towards an extended taxonomy of information dynamics via Integrated Information Decompositioncited by 4
- 2025Impact of Layer Norm on Memorization and Generalization in Transformerscited by 3
- 2024Evolving reservoir computers reveals bidirectional coupling between predictive power and emergent dynamicscited by 1
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
- 2026Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuitscited by 9
- 2026Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Modelscited by 0
- 2026Unveiling the Reasoning Process of Large Language Modelscited by 0
- 2026Information Dynamics of Language Communicationcited by 0
- 2026From cacophony to hierarchy: a principled framework for assessing AI consciousnesscited by 0
- 2026From cacophony to hierarchy: a principled framework for assessing AI consciousness28 Sep 2026
- 2026Verbalizable Representations Form a Global Workspace in Language Models16 Jul 2026
- 2026Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits2 Jul 2026
- 2026Information Dynamics of Language Communication29 Jun 2026
- 2026Unveiling the Reasoning Process of Large Language Models31 Mar 2026
- 2026Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models7 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2026Modular Cognitive Architecture Emerges in Large Language Models
- 2021A brain basis of dynamical intelligence for AI and computational neuroscience
- 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems
- 2025The brain-AI convergence: Predictive and generative world models for general-purpose computation
- 2026Remapping and navigation of an embedding space via error minimization: a fundamental organizational principle of cognition in natural and artificial systems
- 2024The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Units
- 2025Reinforcement Learning Fine-Tuning Enhances Activation Intensity and Diversity in the Internal Circuitry of LLMs
- 2026From Signals to Structure: How Memory Architecture Drives Language Emergence in LLM Agents