mlsanityData to 14 Oct 2026

The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models

arXiv 2609.10947 ↗First posted 10 Sep 2026q-bio.NC cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Multimodal large language models predict brain activity, but brain alignment has been a measurement, not a design tool. We propose the Platonic brain bridge hypothesis: omni models, multimodal large language models that process video, audio and text jointly, converge on brain-like representations usable in both directions. From model to brain, brain-likeness of seven omni models is stable across participants, rises with every input channel in three bases, and our encoders lead the Algonauts 2025 out-of-distribution leaderboard. From brain to model, Brain-MoE fixes the expert partition of a frozen base to the seven networks of human cortex, trains experts on network-labelled Brain-AVQA questions, raises held-out accuracy in all 15 model-benchmark pairs by 6.42 percentage points on average and exceeds capacity-matched random experts in 14. Brain-Scope localizes the correspondence to sparse features whose removal weakens brain prediction. Human brain organization is therefore a usable architectural prior for multimodal large language models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 44 of its 68 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

44 of 68 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction30 Apr
  2. 2024OmniBench: Towards The Future of Universal Omni-Language Models23 Sep
  3. 2024Human-like object concept representations emerge naturally in multimodal large language models1 Jul

Other references · 41

  1. 2023Large Language Modelscited by 780
  2. 2024Scaling and evaluating sparse autoencoderscited by 553
  3. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
  4. 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 193
  5. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 191
  6. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 155
  7. 2019Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)cited by 128
  8. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalitiescited by 110
  9. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
  10. 2025MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligencecited by 61
  11. 2024MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Modelscited by 44
  12. 2019Learning From Brains How to Regularize Machinescited by 38
29 more
  1. 2024Contextual Feature Extraction Hierarchies Converge in Large Language Models and the Braincited by 32
  2. 2026Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligencecited by 30
  3. 2023Tackling Data Bias in MUSIC-AVQA: Crafting a Balanced Dataset for Unbiased Question-Answeringcited by 29
  4. 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 21
  5. 2024The Algonauts Project 2025 Challenge: How the Human Brain Makes Sense of Multimodal Moviescited by 20
  6. 2025TRIBE: TRImodal Brain Encoder for whole-brain fMRI response predictioncited by 19
  7. 2026Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scalecited by 16
  8. 2026A foundation model of vision, audition, and language for in-silico neurosciencecited by 13
  9. 2025Mixture of Cognitive Reasoners: Modular Reasoning with Brain-Like Specializationcited by 9
  10. 2025MoRE-Brain: Routed Mixture of Experts for Interpretable and Generalizable Cross-Subject fMRI Visual Decodingcited by 7
  11. 2025Brain-Language Model Alignment: Insights into the Platonic Hypothesis and Intermediate-Layer Advantagecited by 5
  12. 2025Insights from the Algonauts 2025 Winnerscited by 4
  13. 2025Brain-tuning Improves Generalizability and Efficiency of Brain Alignment in Speech Modelscited by 4
  14. 2026Beyond representational alignment with brain-guided language models for robust reasoningcited by 4
  15. 2026Inside the Visual Mind: Neuroscience-Motivated Concept Circuits for Interpreting and Steering Vision Transformerscited by 3
  16. 2025Brain-Like Processing Pathways Form in Models With Heterogeneous Expertscited by 2
  17. 2026Local Intrinsic Dimension of Representations Predicts Alignment and Generalization in AI Models and Human Braincited by 2
  18. 2026Abstraction Induces the Brain Alignment of Language and Speech Modelscited by 2
  19. 2026When Language Models Lose Their Mind: The Consequences of Brain Misalignmentcited by 2
  20. 2026Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topographycited by 2
  21. 2026Discovering Functionally Selective Brain Regions with a Deep Topographic Multimodal Modelcited by 2
  22. 2026TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streamingcited by 2
  23. 2025Probing Multimodal Fusion in the Brain: The Dominance of Audiovisual Streams in Naturalistic Encodingcited by 1
  24. 2025Topoformer: brain-like topographic organization in Transformer language models through spatial querying and reweightingcited by 1
  25. 2025The One Where They Brain-Tune for Social Cognition: Multi-Modal Brain-Tuning on Friendscited by 1
  26. 2026Do Language Models Align with Brains? Prediction Scores Are Not Enoughcited by 1
  27. 2026FPED: A Functional-Network Prior-Guided Mixture-of-Experts Framework for Interpretable Brain Decodingcited by 1
  28. 2026MIRAGE: Adaptive Multimodal Gating for Whole-Brain fMRI Encodingcited by 1
  29. 2026Interpreting Brain Responses to Language with Sparse Features from Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Decoding the Multimodal Mind: Generalizable Brain-to-Text Translation via Multimodal Alignment and Adaptive Routing
  2. 2025Do Large Language Models Think Like the Brain? Sentence-Level Evidences from Layer-Wise Embeddings and fMRI
  3. 2026Linguistic properties and model scale in brain encoding: from small to compressed language models
  4. 2025Uncovering Brain-Like Hierarchical Patterns in Vision-Language Models through fMRI-Based Neural Encoding
  5. 2026Cognitive Expert Language Models Better Align with the Corresponding Brain Systems
  6. 2026Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models
  7. 2026Computational Lesions in Multilingual Language Models Separate Shared and Language-specific Brain Alignment
  8. 2026Mind-Omni: A Unified Multi-Task Framework for Brain-Vision-Language Modeling via Discrete Diffusion