mlsanityData to 14 Oct 2026

Human Implicit Preference-Based Policy Fine-tuning for Multi-Agent Reinforcement Learning in USV Swarm

arXiv 2503.03796 ↗First posted 5 Mar 2025cs.MA cs.AI cs.LGcited by 6

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

also in 1 more
Abstract (arXiv)

Multi-Agent Reinforcement Learning (MARL) has shown promise in solving complex problems involving cooperation and competition among agents, such as an Unmanned Surface Vehicle (USV) swarm used in search and rescue, surveillance, and vessel protection. However, aligning system behavior with user preferences is challenging due to the difficulty of encoding expert intuition into reward functions. To address the issue, we propose a Reinforcement Learning with Human Feedback (RLHF) approach for MARL that resolves credit-assignment challenges through an Agent-Level Feedback system categorizing feedback into intra-agent, inter-agent, and intra-team types. To overcome the challenges of direct human feedback, we employ a Large Language Model (LLM) evaluator to validate our approach using feedback scenarios such as region constraints, collision avoidance, and task allocation. Our method effectively refines USV swarm policies, addressing key challenges in multi-agent systems while maintaining fairness and performance consistency.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 31 of its 57 references that are in the map. First posted on arXiv 5 Mar 2025.

Builds on

31 of 57 references.

References · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2020Learning to summarize from human feedbackcited by 2,588
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  6. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  7. 2018Learning Dexterous In-Hand Manipulationcited by 687
  8. 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 328
  9. 2018Reward learning from human preferences and demonstrations in Ataricited by 316
  10. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 302
  11. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
  12. 2017Inverse Reward Designcited by 249
19 more
  1. 2018Optimization of Molecules via Deep Reinforcement Learningcited by 200
  2. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 173
  3. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 129
  4. 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 87
  5. 2023Preference Transformer: Modeling Human Preferences using Transformers for RLcited by 79
  6. 2020Reinforcement Learning for Autonomous Driving with Latent State Inference and Spatial-Temporal Relationshipscited by 37
  7. 2021RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecastingcited by 36
  8. 2020Avoiding Side Effects in Complex Environmentscited by 24
  9. 2024Multi-Agent Dynamic Relational Reasoning for Social Robot Navigationcited by 20
  10. 2021Autonomous Driving Strategies at Intersections: Scenarios, State-of-the-Art, and Future Outlookscited by 15
  11. 2022Coordination with Humans via Strategy Matchingcited by 14
  12. 2021Cooperative Assistance in Robotic Surgery through Multi-Agent Reinforcement Learningcited by 11
  13. 2023Learning a Universal Human Prior for Dexterous Manipulation from Human Preferencecited by 10
  14. 2023Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimationcited by 10
  15. 2023Robust Unmanned Surface Vehicle Navigation with Distributional Reinforcement Learningcited by 8
  16. 2025Preference VLM: Leveraging VLMs for Scalable Preference-Based Reinforcement Learningcited by 7
  17. 2023Robust Driving Policy Learning with Guided Meta Reinforcement Learningcited by 5
  18. 2024Robust Offline Imitation Learning from Diverse Auxiliary Datacited by 5
  19. 2024Importance Sampling-Guided Meta-Training for Intelligent Agents in Highly Interactive Environmentscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  2. 2025Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handlingcited by 6
  3. 2026Priority-Aware Multi-Robot Coverage Path Planningcited by 2
  4. 2026Cooperative Informative Sensing for Monitoring Dynamic Indoor Environments via Multi-Agent Reinforcement Learningcited by 0
  5. 2026Multi-Agent Reinforcement Learning via Agent-Specific Preferencecited by 0
  6. 2026Hierarchical Residual Policy Learning for Real-World Mobile Manipulation with Sparse Human Guidancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025An Extended Benchmarking of Multi-Agent Reinforcement Learning Algorithms in Complex Fully Cooperative Tasks
  2. 2025M3HF: Multi-agent Reinforcement Learning from Multi-phase Human Feedback of Mixed Quality
  3. 2025A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives
  4. 2025Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Survey
  5. 2024Multiple Ships Cooperative Navigation and Collision Avoidance using Multi-agent Reinforcement Learning with Communication ↗
  6. 2024Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework
  7. 2025CTC: The Composite Task Challenge for Cooperative Multi-Agent Reinforcement Learning
  8. 2025A Learning Framework For Cooperative Collision Avoidance of UAV Swarms Leveraging Domain Knowledge