Robust Instruction Compliance in Cooperative Multi-Agent Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Multi-agent reinforcement learning (MARL) in real-world use cases may need to adapt to external natural language instructions that interrupt ongoing behavior and conflict with long-horizon objectives. However, conditioning rewards on instructions introduces a fundamental failure mode as Bellman updates couple value estimates across instruction contexts, leading to inconsistent values when instructions interrupt macro-actions. We propose Macro-Action Value Correction for Instruction Compliance (MAVIC), which corrects Bellman backups at instruction boundaries by correcting the incoming instruction objective and restoring the continuation value under the current objective. Unlike reward shaping, MAVIC modifies the bootstrapping target itself, enabling consistent value estimation under stochastic instruction switching within a unified policy. We provide theoretical analysis and an actor-critic implementation, and show that MAVIC achieves high instruction compliance while preserving base task performance in increasingly complex cooperative multi-agent environments.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 24 references that are in the map. First posted on arXiv 12 May 2026.
Builds on
19 of 24 references.
References · 19
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
- 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 221
- 2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 156
- 2024Yell At Your Robot: Improving On-the-Fly from Language Correctionscited by 124
- 2023"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomycited by 60
- 2023Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environmentscited by 53
- 2023Interactive Robot Learning from Verbal Correctioncited by 29
- 2021Unbiased Asymmetric Reinforcement Learning under Partial Observabilitycited by 25
- 2024SleeperNets: Universal Backdoor Poisoning Attacks Against Reinforcement Learning Agentscited by 19
- 2022Asynchronous Actor-Critic for Multi-Agent Reinforcement Learningcited by 18
7 more
- 2023Asynchronous Multi-Agent Reinforcement Learning for Efficient Real-Time Multi-Robot Cooperative Explorationcited by 18
- 2021Coordinated Proximal Policy Optimizationcited by 16
- 2015Decentralized Control of Partially Observable Markov Decision Processes using Belief Space Macro-actionscited by 11
- 2018FollowNet: Robot Navigation by Following Natural Language Directions with Deep Reinforcement Learningcited by 10
- 2024Learning Efficient and Robust Language-conditioned Manipulation using Textual-Visual Relevancy and Equivariant Language Mappingcited by 10
- 2024PREDILECT: Preferences Delineated with Zero-Shot Language-based Reasoning in Reinforcement Learningcited by 8
- 2014Planning for Decentralized Control of Multiple Robots Under Uncertaintycited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025ICCO: Learning an Instruction-conditioned Coordinator for Language-guided Task-aligned Multi-robot Control
- 2025Agentic Policy Optimization via Instruction-Policy Co-Evolution
- 2026Regime-Conditional Stabilisation of LLM-Augmented Cooperative Multi-Agent Reinforcement Learning
- 2025M3HF: Multi-agent Reinforcement Learning from Multi-phase Human Feedback of Mixed Quality
- 2026Adaptive Robust Estimator for Multi-Agent Reinforcement Learning
- 2025Automata-Conditioned Cooperative Multi-Agent Reinforcement Learning
- 2026Retaining Suboptimal Actions to Follow Shifting Optima in Multi-Agent Reinforcement Learning
- 2026Distributionally Robust Cooperative Multi-Agent Reinforcement Learning via Robust Value Factorization