mlsanityData to 14 Oct 2026

Neural Brain: A Neuroscience-inspired Framework for Embodied Agents

arXiv 2505.07634 ↗First posted 12 May 2025cs.RO cs.AI cs.CVcited by 10

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

The rapid evolution of artificial intelligence (AI) has shifted from static, data-driven models to dynamic systems capable of perceiving and interacting with real-world environments. Despite advancements in pattern recognition and symbolic reasoning, current AI systems, such as large language models, remain disembodied, unable to physically engage with the world. This limitation has driven the rise of embodied AI, where autonomous agents, such as humanoid robots, must navigate and manipulate unstructured environments with human-like adaptability. At the core of this challenge lies the concept of Neural Brain, a central intelligence system designed to drive embodied agents with human-like adaptability. A Neural Brain must seamlessly integrate multimodal sensing and perception with cognitive capabilities. Achieving this also requires an adaptive memory system and energy-efficient hardware-software co-design, enabling real-time action in dynamic environments. This paper introduces a unified framework for the Neural Brain of embodied agents, addressing two fundamental challenges: (1) defining the core components of Neural Brain and (2) bridging the gap between static AI models and the dynamic adaptability required for real-world deployment. To this end, we propose a biologically inspired architecture that integrates multimodal active sensing, perception-cognition-action function, neuroplasticity-based memory storage and updating, and neuromorphic hardware/software optimization. Furthermore, we also review the latest research on embodied agents across these four aspects and analyze the gap between current AI systems and human intelligence. By synthesizing insights from neuroscience, we outline a roadmap towards the development of generalizable, autonomous agents capable of human-level intelligence in real-world scenarios.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 163 of its 498 references that are in the map. First posted on arXiv 12 May 2025.

Builds on

163 of 498 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents13 Feb
  2. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents21 Jan
  3. 2024KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systems23 Sep
  4. 2024AeroVerse: UAV-Agent Benchmark Suite for Simulating, Pre-training, Finetuning, and Evaluating Aerospace Embodied World Models28 Aug
  5. 2024GenRL: Multimodal-foundation world models for generalization in embodied agents26 Jun
  6. 2024Agent AI: Surveying the Horizons of Multimodal Interaction7 Jan
  7. 2023Building Open-Ended Embodied Agent via Language-Policy Bidirectional Adaptation12 Dec
  8. 2022JARVIS: A Neuro-Symbolic Commonsense Reasoning Framework for Conversational Embodied Agents28 Aug

Other references · 155

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  8. 2024GPT-4o System Cardcited by 4,588
  9. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  10. 2023Qwen Technical Reportcited by 3,344
  11. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  12. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
38 more
  1. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  4. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  5. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  7. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  8. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  9. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  10. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  11. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  12. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  13. 2023PaLM 2 Technical Reportcited by 1,162
  14. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  15. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  16. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  17. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  18. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  19. 2024DeepSeek-V3 Technical Reportcited by 811
  20. 2022A Generalist Agentcited by 808
  21. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
  22. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  23. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
  24. 2014Neural Turing Machinescited by 644
  25. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  26. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  27. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
  28. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
  29. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  30. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
  31. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  32. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 349
  33. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 346
  34. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  35. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 315
  36. 2022UL2: Unifying Language Learning Paradigmscited by 295
  37. 2022AudioGen: Textually Guided Audio Generationcited by 295
  38. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 282

50 most cited of 155 All 155 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 6.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026NeuroVLA: A Brain-inspired Embodied Intelligence for Fluid and Fast Reflexive Robotics Control
  2. 2025RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systems
  3. 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence
  4. 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI
  5. 2025Towards Neurocognitive-Inspired Intelligence: From AI's Structural Mimicry to Human-Like Functional Cognition
  6. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Models
  7. 2026BrainMem: Brain-Inspired Evolving Memory for Embodied Agent Task Planning
  8. 2026Athena-Brain Technical Report: An Efficient Robot Brain for General Intelligence and Embodied Interaction