mlsanityData to 7 Oct 2026

RoCo: Dialectic Multi-Robot Collaboration with Large Language Models

arXiv 2307.04738 ↗First posted 10 Jul 2023cs.RO cs.AI cs.LGcited by 171

Not on the map. Why?

Cited by 92 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a novel approach to multi-robot collaboration that harnesses the power of pre-trained large language models (LLMs) for both high-level communication and low-level path planning. Robots are equipped with LLMs to discuss and collectively reason task strategies. They then generate sub-task plans and task space waypoint paths, which are used by a multi-arm motion planner to accelerate trajectory planning. We also provide feedback from the environment, such as collision checking, and prompt the LLM agents to improve their plan and waypoints in-context. For evaluation, we introduce RoCoBench, a 6-task benchmark covering a wide range of multi-robot collaboration scenarios, accompanied by a text-only dataset for agent representation and reasoning. We experimentally demonstrate the effectiveness of our approach -- it achieves high success rates across all tasks in RoCoBench and adapts to variations in task semantics. Our dialog setup offers high interpretability and flexibility -- in real world experiments, we show RoCo easily incorporates human-in-the-loop, where a user can communicate and collaborate with a robot agent to complete tasks together. See project website https://project-roco.github.io for videos and code.

Key facts. Cited by 171 papers in the map (data through 7 Oct 2026). Builds on 35 of its 64 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

35 of 64 references.

References · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  5. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  7. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  8. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  9. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  10. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  11. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  12. 2018AI safety via debatecited by 384
23 more
  1. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
  2. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  3. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 303
  4. 2023Reward Design with Language Modelscited by 257
  5. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 223
  6. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 219
  7. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  8. 2023Task and Motion Planning with Large Language Models for Object Rearrangementcited by 140
  9. 2022Language Models as Agent Modelscited by 131
  10. 2023Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Modelcited by 115
  11. 2023AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkerscited by 111
  12. 2023"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomycited by 60
  13. 2022Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Modelscited by 58
  14. 2023DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agentscited by 52
  15. 2023Clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agentscited by 52
  16. 2023Chat with the Environment: Interactive Multimodal Perception Using Large Language Modelscited by 42
  17. 2023Demo2Code: From Summarizing Demonstrations to Synthesizing Code via Extended Chain-of-Thoughtcited by 39
  18. 2022Learning Dexterous Manipulation from Exemplar Object Trajectories and Pre-Graspscited by 27
  19. 2023AlphaBlock: Embodied Finetuning for Vision-Language Reasoning in Robot Manipulationcited by 19
  20. 2020Learning a Decentralized Multi-arm Motion Plannercited by 17
  21. 2023Dialogue Games for Benchmarking Language Understanding: Motivation, Taxonomy, Strategycited by 12
  22. 2016Closed-Chain Manipulation of Large Objects by Multi-Arm Robotic Systemscited by 6
  23. 2022Visually Grounded Task and Motion Planning for Mobile Manipulationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 56; 2025: 59; 2026: 40.

171 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  3. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  4. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  5. 2023A Comprehensive Overview of Large Language Modelscited by 639
  6. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  7. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 235
  8. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
  9. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 148
  10. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  11. 2023Large Language Models for Robotics: A Surveycited by 132
  12. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
38 more
  1. 2024Empowering Biomedical Discovery with AI Agentscited by 117
  2. 2024FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Makingcited by 112
  3. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 102
  4. 2024AIOS: LLM Agent Operating Systemcited by 94
  5. 2023On Generative Agents in Recommendationcited by 88
  6. 2024Embodied LLM Agents Learn to Cooperate in Organized Teamscited by 81
  7. 2024Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligencecited by 80
  8. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  9. 2024Self-Evolving Multi-Agent Collaboration Networks for Software Developmentcited by 73
  10. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  11. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  12. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planningcited by 59
  13. 2024Agents' Room: Narrative Generation through Multi-step Collaborationcited by 57
  14. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  15. 2025MARFT: Multi-Agent Reinforcement Fine-Tuningcited by 51
  16. 2023Multi-Agent Consensus Seeking via Large Language Modelscited by 50
  17. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49
  18. 2024Towards Efficient LLM Grounding for Embodied Multi-Agent Collaborationcited by 45
  19. 2024COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Modelscited by 43
  20. 2024Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent Systemcited by 40
  21. 2023LLMR: Real-time Prompting of Interactive Worlds using Large Language Modelscited by 39
  22. 2023Controlling Large Language Model-based Agents for Large-Scale Decision-Making: An Actor-Critic Approachcited by 38
  23. 2026Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agentscited by 37
  24. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  25. 2024LLaMAR: Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environmentscited by 32
  26. 2024LaMMA-P: Generalizable Multi-Agent Long-Horizon Task Allocation and Planning with LM-Driven PDDL Plannercited by 31
  27. 2025Internet of Agents: Fundamentals, Applications, and Challengescited by 31
  28. 2024Exploring Autonomous Agents through the Lens of Large Language Models: A Reviewcited by 30
  29. 2024TwoStep: Multi-agent Task Planning using Classical Planners and Large Language Modelscited by 29
  30. 2025Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulationscited by 28
  31. 2025RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaborationcited by 28
  32. 2023Interactive Planning Using Large Language Models for Partially Observable Robotics Taskscited by 27
  33. 2025Large Language Models for Multi-Robot Systems: A Surveycited by 26
  34. 2025Large Language Models Miss the Multi-Agent Markcited by 25
  35. 2024Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigationcited by 24
  36. 2024EMOS: Embodiment-aware Heterogeneous Multi-robot Operating System with LLM Agentscited by 24
  37. 2024S-Agents: Self-organizing Agents in Open-ended Environmentscited by 22
  38. 2024Kolb-Based Experiential Learning for Generalist Agents with Human-Level Kaggle Data Science Performancecited by 22

50 most cited of 171 All 171 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Tool-RoCo: An Agent-as-Tool Self-organization Large Language Model Benchmark in Multi-robot Cooperation
  2. 2026DynaHMRC: Decentralized Heterogeneous Multi-Robot Collaboration for Dynamic Tasks with Large Language Models ↗
  3. 2025Leveraging Adaptive Group Negotiation for Heterogeneous Multi-Robot Collaboration with Large Language Models
  4. 2026MeCo: Enhancing LLM-Empowered Multi-Robot Collaboration via Similar Task Memoization ↗
  5. 2026Scale-Plan: Scalable Language-Enabled Task Planning for Heterogeneous Multi-Robot Teams
  6. 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models
  7. 2026STEP: State-Aware Task Estimation and Planning with Multi-Modal LLMs for Human-Robot Collaboration
  8. 2023LLM-MARS: Large Language Model for Behavior Tree Generation and NLP-enhanced Dialogue in Multi-Agent Robot Systems