# A Brain-like Synergistic Core in LLMs Drives Behaviour and Learning

arXiv: https://arxiv.org/abs/2601.06851 · HTML: https://mlsanity.com/paper/2601.06851/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 11 Jan 2026.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://mlsanity.com/paper/2409.12122/) (2024)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training](https://mlsanity.com/paper/2501.17161/) (2025)
- [ShortGPT: Layers in Large Language Models are More Redundant Than You Expect](https://mlsanity.com/paper/2403.03853/) (2024)
- [Layer by Layer: Uncovering Hidden Representations in Language Models](https://mlsanity.com/paper/2502.02013/) (2025)
- [OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data](https://mlsanity.com/paper/2410.01560/) (2024)
- [Transformer Layers as Painters](https://mlsanity.com/paper/2407.09298/) (2024)
- [Understanding Convolutional Neural Networks with Information Theory: An Initial Exploration](https://mlsanity.com/paper/1804.06537/) (2018)
- [Intermediate Layer Classifiers for OOD generalization](https://mlsanity.com/paper/2504.05461/) (2025)
- [A Measure of the Complexity of Neural Representations based on Partial Information Decomposition](https://mlsanity.com/paper/2209.10438/) (2022)
- [Learning to Skip the Middle Layers of Transformers](https://mlsanity.com/paper/2506.21103/) (2025)
- [Multivariate Information Theory Uncovers Synergistic Subsystems of the Human Cerebral Cortex](https://mlsanity.com/paper/2206.06477/) (2022)
- [Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks](https://mlsanity.com/paper/2210.02996/) (2022)
- [Partial entropy decomposition reveals higher-order structures in human brain activity](https://mlsanity.com/paper/2301.05307/) (2023)
- [A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functions](https://mlsanity.com/paper/2306.02149/) (2023)
- [Layerwise Importance Analysis of Feed-Forward Networks in Transformer-based Language Models](https://mlsanity.com/paper/2508.17734/) (2025)
- [Integrated information as a common signature of dynamical and information-processing complexity](https://mlsanity.com/paper/2106.10211/) (2021)
- [Towards an extended taxonomy of information dynamics via Integrated Information Decomposition](https://mlsanity.com/paper/2109.13186/) (2021)
- [Impact of Layer Norm on Memorization and Generalization in Transformers](https://mlsanity.com/paper/2511.10566/) (2025)
- [Evolving reservoir computers reveals bidirectional coupling between predictive power and emergent dynamics](https://mlsanity.com/paper/2406.19201/) (2024)

## Built on by (most cited first, 6 of 6)

- [Verbalizable Representations Form a Global Workspace in Language Models](https://mlsanity.com/paper/2607.15495/) (2026)
- [Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits](https://mlsanity.com/paper/2607.01940/) (2026)
- [Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models](https://mlsanity.com/paper/2602.07539/) (2026)
- [Unveiling the Reasoning Process of Large Language Models](https://mlsanity.com/paper/2603.29735/) (2026)
- [Information Dynamics of Language Communication](https://mlsanity.com/paper/2606.30096/) (2026)
- [From cacophony to hierarchy: a principled framework for assessing AI consciousness](https://mlsanity.com/paper/2609.35618/) (2026)

## Abstract (arXiv)

The independent evolution of intelligence in biological and artificial systems offers a unique opportunity to identify its fundamental computational principles. Here we show that large language models spontaneously develop synergistic cores -- components where information integration exceeds individual parts -- remarkably similar to those in the human brain. Using principles of information decomposition across multiple LLM model families and architectures, we find that areas in middle layers exhibit synergistic processing while early and late layers rely on redundancy, mirroring the informational organisation in biological brains. This organisation emerges through learning and is absent in randomly initialised networks. Crucially, ablating synergistic components causes disproportionate behavioural changes and performance loss, aligning with theoretical predictions about the fragility of synergy. Moreover, fine-tuning synergistic regions through reinforcement learning yields significantly greater performance gains than training redundant components, yet supervised fine-tuning shows no such advantage. This convergence suggests that synergistic information processing is a fundamental property of intelligence, providing targets for principled model design and testable predictions for biological intelligence.