# Fast Teammate Adaptation in the Presence of Sudden Policy Change

arXiv: https://arxiv.org/abs/2305.05911 · HTML: https://mlsanity.com/paper/2305.05911/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2023.

## Built on by (most cited first, 8 of 8)

- [Mutual Theory of Mind in Human-AI Collaboration: An Empirical Study with LLM-driven AI Agents in a Real-time Shared Workspace Task](https://mlsanity.com/paper/2409.08811/) (2024)
- [Efficient Human-AI Coordination via Preparatory Language-based Convention](https://mlsanity.com/paper/2311.00416/) (2023)
- [Fast Peer Adaptation with Context-aware Exploration](https://mlsanity.com/paper/2402.02468/) (2024)
- [Beyond Shallow Behavior: Task-Efficient Value-Based Multi-Task Offline MARL via Skill Discovery](https://mlsanity.com/paper/2502.08985/) (2025)
- [TransAM: Transformer-Based Agent Modeling for Multi-Agent Systems via Local Trajectory Encoding](https://mlsanity.com/paper/2508.02826/) (2025)
- [From Solo to Symphony: Orchestrating Multi-Agent Collaboration with Single-Agent Demos](https://mlsanity.com/paper/2511.02762/) (2025)
- [FlickerFusion: Intra-trajectory Domain Generalizing Multi-Agent RL](https://mlsanity.com/paper/2410.15876/) (2024)
- [Online Change-point Detection for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2609.05298/) (2026)

## Abstract (arXiv)

In cooperative multi-agent reinforcement learning (MARL), where an agent coordinates with teammate(s) for a shared goal, it may sustain non-stationary caused by the policy change of teammates. Prior works mainly concentrate on the policy change during the training phase or teammates altering cross episodes, ignoring the fact that teammates may suffer from policy change suddenly within an episode, which might lead to miscoordination and poor performance as a result. We formulate the problem as an open Dec-POMDP, where we control some agents to coordinate with uncontrolled teammates, whose policies could be changed within one episode. Then we develop a new framework, fast teammates adaptation (Fastap), to address the problem. Concretely, we first train versatile teammates' policies and assign them to different clusters via the Chinese Restaurant Process (CRP). Then, we train the controlled agent(s) to coordinate with the sampled uncontrolled teammates by capturing their identifications as context for fast adaptation. Finally, each agent applies its local information to anticipate the teammates' context for decision-making accordingly. This process proceeds alternately, leading to a robust policy that can adapt to any teammates during the decentralized execution phase. We show in multiple multi-agent benchmarks that Fastap can achieve superior performance than multiple baselines in stationary and non-stationary scenarios.