Neural Brain: A Neuroscience-inspired Framework for Embodied Agents
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
The rapid evolution of artificial intelligence (AI) has shifted from static, data-driven models to dynamic systems capable of perceiving and interacting with real-world environments. Despite advancements in pattern recognition and symbolic reasoning, current AI systems, such as large language models, remain disembodied, unable to physically engage with the world. This limitation has driven the rise of embodied AI, where autonomous agents, such as humanoid robots, must navigate and manipulate unstructured environments with human-like adaptability. At the core of this challenge lies the concept of Neural Brain, a central intelligence system designed to drive embodied agents with human-like adaptability. A Neural Brain must seamlessly integrate multimodal sensing and perception with cognitive capabilities. Achieving this also requires an adaptive memory system and energy-efficient hardware-software co-design, enabling real-time action in dynamic environments. This paper introduces a unified framework for the Neural Brain of embodied agents, addressing two fundamental challenges: (1) defining the core components of Neural Brain and (2) bridging the gap between static AI models and the dynamic adaptability required for real-world deployment. To this end, we propose a biologically inspired architecture that integrates multimodal active sensing, perception-cognition-action function, neuroplasticity-based memory storage and updating, and neuromorphic hardware/software optimization. Furthermore, we also review the latest research on embodied agents across these four aspects and analyze the gap between current AI systems and human intelligence. By synthesizing insights from neuroscience, we outline a roadmap towards the development of generalizable, autonomous agents capable of human-level intelligence in real-world scenarios.
Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 163 of its 498 references that are in the map. First posted on arXiv 12 May 2025.
Builds on
163 of 498 references.
Closest ancestors · 8
Same direction, newest first
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents13 Feb
- 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents21 Jan
- 2024KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systems23 Sep
- 2024AeroVerse: UAV-Agent Benchmark Suite for Simulating, Pre-training, Finetuning, and Evaluating Aerospace Embodied World Models28 Aug
- 2024GenRL: Multimodal-foundation world models for generalization in embodied agents26 Jun
- 2024Agent AI: Surveying the Horizons of Multimodal Interaction7 Jan
- 2023Building Open-Ended Embodied Agent via Language-Policy Bidirectional Adaptation12 Dec
- 2022JARVIS: A Neuro-Symbolic Commonsense Reasoning Framework for Conversational Embodied Agents28 Aug
Other references · 155
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023GPT-4 Technical Reportcited by 18,396
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2024GPT-4o System Cardcited by 4,588
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2023Qwen Technical Reportcited by 3,344
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
38 more
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
- 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
- 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
- 2023PaLM 2 Technical Reportcited by 1,162
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
- 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
- 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
- 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
- 2024DeepSeek-V3 Technical Reportcited by 811
- 2022A Generalist Agentcited by 808
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
- 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
- 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
- 2014Neural Turing Machinescited by 644
- 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
- 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
- 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
- 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
- 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
- 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 349
- 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 346
- 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
- 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 315
- 2022UL2: Unifying Language Learning Paradigmscited by 295
- 2022AudioGen: Textually Guided Audio Generationcited by 295
- 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 282
50 most cited of 155 All 155 (JSON)
Built on by
Citing papers per year: 2025: 4; 2026: 6.
10 papers.
- 2026FadeMem: Biologically-Inspired Forgetting for Efficient Agent Memorycited by 14
- 2025Toward Embodied AGI: A Review of Embodied AI and the Road Aheadcited by 8
- 2025Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspectivecited by 7
- 2026Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topographycited by 2
- 2026Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Actioncited by 2
- 2025HiBerNAC: Hierarchical Brain-inspired Robotic Neural Agent Collective for Disentangling Complex Manipulationcited by 1
- 2026UniFS: Unified Fast-to-Slow Hierarchical Architecture for Vision-Language-Action Modelscited by 1
- 2025Physics-Informed Spiking Neural Networks via Conservative Flux Quantizationcited by 0
- 2026A New Strategy for Artificial Intelligence: Training Foundation Models Directly on Human Brain Datacited by 0
- 2026Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branchescited by 0
- 2026UniFS: Unified Fast-to-Slow Hierarchical Architecture for Vision-Language-Action Models22 Jun 2026
- 2026Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branches31 May 2026
- 2026Sparse Autoencoders Map Brain-LLM Alignment onto Cortical Semantic Topography21 May 2026
- 2026Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action5 Apr 2026
- 2026FadeMem: Biologically-Inspired Forgetting for Efficient Agent Memory26 Jan 2026
- 2026A New Strategy for Artificial Intelligence: Training Foundation Models Directly on Human Brain Data17 Jan 2026
- 2025Physics-Informed Spiking Neural Networks via Conservative Flux Quantization26 Nov 2025
- 2025Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspective11 Sep 2025
- 2025HiBerNAC: Hierarchical Brain-inspired Robotic Neural Agent Collective for Disentangling Complex Manipulation9 Jun 2025
- 2025Toward Embodied AGI: A Review of Embodied AI and the Road Ahead20 May 2025
Similar, not cited
Close in wording, no citation either way
- 2026NeuroVLA: A Brain-inspired Embodied Intelligence for Fluid and Fast Reflexive Robotics Control
- 2025RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systems
- 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence
- 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI
- 2025Towards Neurocognitive-Inspired Intelligence: From AI's Structural Mimicry to Human-Like Functional Cognition
- 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Models
- 2026BrainMem: Brain-Inspired Evolving Memory for Embodied Agent Task Planning
- 2026Athena-Brain Technical Report: An Efficient Robot Brain for General Intelligence and Embodied Interaction