# Multi-Agent Systems for Robotic Autonomy with LLMs

arXiv: https://arxiv.org/abs/2505.05762 · HTML: https://mlsanity.com/paper/2505.05762/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 51 references that are in the map.
- First posted on arXiv 9 May 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 17 of 17)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Guiding Pretraining in Reinforcement Learning with Large Language Models](https://mlsanity.com/paper/2302.06692/) (2023)
- [SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models](https://mlsanity.com/paper/2309.10062/) (2023)
- [AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkers](https://mlsanity.com/paper/2306.06531/) (2023)
- [Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?](https://mlsanity.com/paper/2309.15943/) (2023)
- [ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning](https://mlsanity.com/paper/2308.13724/) (2023)
- [GPT-4V(ision) for Robotics: Multimodal Task Planning from Human Demonstration](https://mlsanity.com/paper/2311.12015/) (2023)
- [Understanding Large-Language Model (LLM)-powered Human-Robot Interaction](https://mlsanity.com/paper/2401.03217/) (2024)
- [TwoStep: Multi-agent Task Planning using Classical Planners and Large Language Models](https://mlsanity.com/paper/2403.17246/) (2024)
- [Large Language Models for Multi-Robot Systems: A Survey](https://mlsanity.com/paper/2502.03814/) (2025)
- [YOLO-MARL: You Only LLM Once for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2410.03997/) (2024)
- [Deep Reinforcement Learning with Enhanced PPO for Safe Mobile Robot Navigation](https://mlsanity.com/paper/2405.16266/) (2024)
- [Natural Multimodal Fusion-Based Human-Robot Interaction: Application With Voice and Deictic Posture via Large Language Model](https://mlsanity.com/paper/2501.00785/) (2025)
- [Automatic MILP Model Construction for Multi-Robot Task Allocation and Scheduling Based on Large Language Models](https://mlsanity.com/paper/2503.13813/) (2025)
- [Imitation Learning for Autonomous Trajectory Learning of Robot Arms in Space](https://mlsanity.com/paper/2008.04007/) (2020)
- [Deep Reinforcement Learning for Decentralized Multi-Robot Control: A DQN Approach to Robustness and Information Integration](https://mlsanity.com/paper/2408.11339/) (2024)

## Built on by (most cited first, 9 of 9)

- [Embracing Trustworthy Brain-Agent Collaboration as Paradigm Extension for Intelligent Assistive Technologies](https://mlsanity.com/paper/2510.22095/) (2025)
- [AgentBalance: Backbone-then-Topology Design for Cost-Effective Multi-Agent Systems under Budget Constraints](https://mlsanity.com/paper/2512.11426/) (2025)
- [A Hierarchical Agentic Framework for Autonomous Drone-Based Visual Inspection](https://mlsanity.com/paper/2510.00259/) (2025)
- [Diffusing to Coordinate: Efficient Online Multi-Agent Diffusion Policies](https://mlsanity.com/paper/2602.18291/) (2026)
- [ROSClaw: A Hierarchical Semantic-Physical Framework for Heterogeneous Multi-Agent Collaboration](https://mlsanity.com/paper/2604.04664/) (2026)
- [Episodic Memory Temporal Consistency for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2606.04492/) (2026)
- [HEART: Coordination of Heterogeneous Expert Agents for Physically Grounded Robotic Task Planning](https://mlsanity.com/paper/2606.25404/) (2026)
- [DART: A DAG-Based Reputation and Incentive Framework via Blockchain-Enabled Governance for Trustworthy LLM Multi-Agent Collaboration](https://mlsanity.com/paper/2609.05529/) (2026)
- [Flowing Faster to Coordinate: One-Step Online Multi-Agent Flow Policies](https://mlsanity.com/paper/2610.01882/) (2026)

## Abstract (arXiv)

Since the advent of Large Language Models (LLMs), various research based on such models have maintained significant academic attention and impact, especially in AI and robotics. In this paper, we propose a multi-agent framework with LLMs to construct an integrated system for robotic task analysis, mechanical design, and path generation. The framework includes three core agents: Task Analyst, Robot Designer, and Reinforcement Learning Designer. Outputs are formatted as multimodal results, such as code files or technical reports, for stronger understandability and usability. To evaluate generalizability comparatively, we conducted experiments with models from both GPT and DeepSeek. Results demonstrate that the proposed system can design feasible robots with control strategies when appropriate task inputs are provided, exhibiting substantial potential for enhancing the efficiency and accessibility of robotic system development in research and industrial applications.