mlsanityData to 14 Oct 2026

ICCO: Learning an Instruction-conditioned Coordinator for Language-guided Task-aligned Multi-robot Control

arXiv 2503.12122 ↗First posted 15 Mar 2025cs.RO cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advances in Large Language Models (LLMs) have permitted the development of language-guided multi-robot systems, which allow robots to execute tasks based on natural language instructions. However, achieving effective coordination in distributed multi-agent environments remains challenging due to (1) misalignment between instructions and task requirements and (2) inconsistency in robot behaviors when they independently interpret ambiguous instructions. To address these challenges, we propose Instruction-Conditioned Coordinator (ICCO), a Multi-Agent Reinforcement Learning (MARL) framework designed to enhance coordination in language-guided multi-robot systems. ICCO consists of a Coordinator agent and multiple Local Agents, where the Coordinator generates Task-Aligned and Consistent Instructions (TACI) by integrating language instructions with environmental states, ensuring task alignment and behavioral consistency. The Coordinator and Local Agents are jointly trained to optimize a reward function that balances task efficiency and instruction following. A Consistency Enhancement Term is added to the learning objective to maximize mutual information between instructions and robot behaviors, further improving coordination. Simulation and real-world experiments validate the effectiveness of ICCO in achieving language-guided task-aligned multi-robot control. The demonstration can be found at https://yanoyoshiki.github.io/ICCO/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 34 references that are in the map. First posted on arXiv 15 Mar 2025.

Builds on

23 of 34 references.

References · 23

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  5. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  6. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  7. 2023RoCo: Dialectic Multi-Robot Collaboration with Large Language Modelscited by 172
  8. 2019Active Domain Randomizationcited by 170
  9. 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Modelscited by 153
  10. 2023Task and Motion Planning with Large Language Models for Object Rearrangementcited by 141
  11. 2023AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkerscited by 113
  12. 2023Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?cited by 106
11 more
  1. 2018Multi-Agent Common Knowledge Reinforcement Learningcited by 31
  2. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 30
  3. 2024LiP-LLM: Integrating Linear Programming and dependency graph with Large Language Models for multi-robot task planningcited by 24
  4. 2020Kimera-Multi: a System for Distributed Multi-Robot Metric-Semantic Simultaneous Localization and Mappingcited by 18
  5. 2024Probabilistically Correct Language-based Multi-Robot Planning using Conformal Predictioncited by 18
  6. 2023Multi-Robot Cooperative Socially-Aware Navigation Using Multi-Agent Reinforcement Learningcited by 17
  7. 2022Reinforcement Learned Distributed Multi-Robot Navigation with Reciprocal Velocity Obstacle Shaped Rewardscited by 16
  8. 2023Resilient and Distributed Multi-Robot Visual SLAM: Datasets, Experiments, and Lessons Learnedcited by 16
  9. 2024Large Language Models for Orchestrating Bimanual Robotscited by 10
  10. 2024Language-Conditioned Offline RL for Multi-Robot Navigationcited by 9
  11. 2024Cooperative Grasping and Transportation using Multi-agent Reinforcement Learning with Ternary Force Representationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Robust Instruction Compliance in Cooperative Multi-Agent Reinforcement Learning
  2. 2026Hierarchical LLM-Based Multi-Agent Framework with Prompt Optimization for Multi-Robot Task Planning
  3. 2024MARLIN: Multi-Agent Reinforcement Learning Guided by Language-Based Inter-Robot Negotiation
  4. 2018Guiding Policies with Language via Meta-Learning
  5. 2023Conditionally Combining Robot Skills using Large Language Models
  6. 2025CRAFT: Coaching Reinforcement Learning Autonomously using Foundation Models for Multi-Robot Coordination Tasks
  7. 2023Bridging Language and Action: A Survey of Language-Conditioned Robot Manipulation
  8. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks