mlsanityData to 14 Oct 2026

Learning "Partner-Aware" Collaborators in Multi-Party Collaboration

arXiv 2510.22462 ↗First posted 26 Oct 2025cs.AIcited by 2

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large Language Models (LLMs) are increasingly being deployed in agentic settings where they act as collaborators with humans. Therefore, it is increasingly important to be able to evaluate their abilities to collaborate effectively in multi-turn, multi-party tasks. In this paper, we build on the AI alignment and safe interruptibility literature to offer novel theoretical insights on collaborative behavior between LLM-driven collaborator agents and an intervention agent. Our goal is to learn an ideal partner-aware collaborator that increases the group's common-ground (CG) alignment on task-relevant propositions-by intelligently collecting information provided in interventions by a partner agent. We show how LLM agents trained using standard RLHF and related approaches are naturally inclined to ignore possibly well-meaning interventions, which makes increasing group common ground non-trivial in this setting. We employ a two-player Modified-Action MDP to examine this suboptimal behavior of standard AI agents, and propose Interruptible Collaborative Roleplayer (ICR)-a novel partner-aware learning algorithm to train CG-optimal collaborators. Experiments on multiple collaborative task environments show that ICR, on average, is more capable of promoting successful CG convergence and exploring more diverse solutions in such tasks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 69 of its 101 references that are in the map. First posted on arXiv 26 Oct 2025.

Builds on

69 of 101 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Collaborate, Deliberate, Evaluate: How LLM Alignment Affects Coordinated Multi-Agent Outcomes7 Sep
  2. 2025Frictional Agent Alignment Framework: Slow Down and Don't Break Things26 May
  3. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 66

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2017Deep reinforcement learning from human preferencescited by 3,912
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  9. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  10. 2015Trust Region Policy Optimizationcited by 2,580
  11. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  12. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
38 more
  1. 2017Decoupled Weight Decay Regularizationcited by 1,831
  2. 2016Concrete Problems in AI Safetycited by 1,642
  3. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  4. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  5. 2017Hindsight Experience Replaycited by 1,136
  6. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  7. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  8. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  9. 2024Self-Rewarding Language Modelscited by 681
  10. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  11. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  12. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  13. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  14. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 499
  15. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 394
  16. 2024Alignment faking in large language modelscited by 360
  17. 2024From r to Q^*: Your Language Model is Secretly a Q-Functioncited by 242
  18. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 238
  19. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 228
  20. 2023Nash Learning from Human Feedbackcited by 224
  21. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
  22. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 197
  23. 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 108
  24. 2024STaR-GATE: Teaching Language Models to Ask Clarifying Questionscited by 104
  25. 2018NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating Systemcited by 98
  26. 2016The Off-Switch Gamecited by 97
  27. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMscited by 96
  28. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 88
  29. 2019Hindsight Credit Assignmentcited by 79
  30. 2023Language models are not naysayers: An analysis of language models on negation benchmarkscited by 74
  31. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Modelscited by 70
  32. 2024Multi-turn Reinforcement Learning from Preference Human Feedbackcited by 67
  33. 2023Language Instructed Reinforcement Learning for Human-AI Coordinationcited by 63
  34. 2025Beyond Reward Hacking: Causal Rewards for Large Language Model Alignmentcited by 42
  35. 2023Honesty Is the Best Policy: Defining and Mitigating AI Deceptioncited by 40
  36. 2019No Press Diplomacy: Modeling Multi-Agent Gameplaycited by 38
  37. 2018Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibilitycited by 32
  38. 2021On the Critical Role of Conventions in Adaptive Human-AI Collaborationcited by 27

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Frictive Policy Optimization for LLMs: Epistemic Intervention, Risk-Sensitive Control, and Reflective Alignmentcited by 2
  2. 2026Flout at Your Own Risk: LLMs Struggle with Pragmatic Cooperativity Under Epistemic Asymmetrycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaboration
  2. 2025Multi-Agent Language Models: Advancing Cooperation, Coordination, and Adaptation
  3. 2026Training LLMs for Multi-Agent Collaboration with Verifiable Information Rewards
  4. 2024Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration
  5. 2026More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration
  6. 2026Partner-Aware Hierarchical Skill Discovery for Robust Human-AI Collaboration
  7. 2025The Alignment Waltz: Jointly Training Agents to Collaborate for Safety
  8. 2026CollabBench: Benchmarking and Unleashing Collaborative Ability of LLMs with Diverse Players via Proactive Engagement