mlsanityData to 7 Oct 2026

Agent as Cerebrum, Controller as Cerebellum: Implementing an Embodied LMM-based Agent on Drones

arXiv 2311.15033 ↗First posted 25 Nov 2023cs.RO cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

In this study, we present a novel paradigm for industrial robotic embodied agents, encapsulating an 'agent as cerebrum, controller as cerebellum' architecture. Our approach harnesses the power of Large Multimodal Models (LMMs) within an agent framework known as AeroAgent, tailored for drone technology in industrial settings. To facilitate seamless integration with robotic systems, we introduce ROSchain, a bespoke linkage framework connecting LMM-based agents to the Robot Operating System (ROS). We report findings from extensive empirical research, including simulated experiments on the Airgen and real-world case study, particularly in individual search and rescue operations. The results demonstrate AeroAgent's superior performance in comparison to existing Deep Reinforcement Learning (DRL)-based agents, highlighting the advantages of the embodied LMM in complex, real-world scenarios.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 25 Nov 2023.

Builds on

25 of 40 references.

References · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  6. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  8. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  9. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  10. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  11. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  12. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
13 more
  1. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  2. 2019Learning Latent Plans from Playcited by 310
  3. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  4. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  5. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  6. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
  7. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  8. 2019V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Controlcited by 85
  9. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learningcited by 70
  10. 2023Large Language Model-Aware In-Context Learning for Code Generationcited by 42
  11. 2016Learning to Push by Grasping: Using multiple tasks for effective learningcited by 36
  12. 2019Improving Data Efficiency of Self-supervised Learning for Robotic Graspingcited by 6
  13. 2023GRID: A Platform for General Robot Intelligence Developmentcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 3.

9 papers.

  1. 2025UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobilitycited by 48
  2. 2025Towards Efficient Online Tuning of VLM Agents via Counterfactual Soft Reinforcement Learningcited by 17
  3. 2025CHAI: Command Hijacking against embodied AIcited by 9
  4. 2024Leveraging YOLO-World and GPT-4V LMMs for Zero-Shot Person Detection and Action Recognition in Drone Imagerycited by 3
  5. 2025Embodied Crowd Countingcited by 1
  6. 2026If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systemscited by 1
  7. 2025LEO-RobotAgent: A General-purpose Robotic Agent for Language-driven Embodied Operatorcited by 0
  8. 2026Beyond Attack Success Rate: Examining Trigger Leakage in Vision-Language Agentic Systemscited by 0
  9. 2026Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Reviewcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration
  2. 2026AeroWeaver: An Embodied-Agent Harness for Weaving Aerial Skills into Distributed, Adaptive Swarm Execution
  3. 2025RoboNeuron: A Middle-Layer Infrastructure for Agent-Driven Orchestration in Embodied AI
  4. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI
  5. 2026Demonstration-Free Robotic Control via LLM Agents
  6. 2026From Digital to Physical: Digital Agents as Autonomous Coaches for Physical Intelligence
  7. 2026HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memory
  8. 2025General-Purpose Aerial Intelligent Agents Empowered by Large Language Models