# Learning Policy Committees for Effective Personalization in MDPs with Diverse Tasks

arXiv: https://arxiv.org/abs/2503.01885 · HTML: https://mlsanity.com/paper/2503.01885/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 26 Feb 2025.

## Builds on (most cited first, 30 of 30)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model](https://mlsanity.com/paper/1907.00953/) (2019)
- [Reward-Free Exploration for Reinforcement Learning](https://mlsanity.com/paper/2002.02794/) (2020)
- [Multi-Task Reinforcement Learning with Context-based Representations](https://mlsanity.com/paper/2102.06177/) (2021)
- [Meta-Gradient Reinforcement Learning](https://mlsanity.com/paper/1805.09801/) (2018)
- [Multi-Task Reinforcement Learning with Soft Modularization](https://mlsanity.com/paper/2003.13661/) (2020)
- [Multi-task Deep Reinforcement Learning with PopArt](https://mlsanity.com/paper/1809.04474/) (2018)
- [Meta-Reinforcement Learning of Structured Exploration Strategies](https://mlsanity.com/paper/1802.07245/) (2018)
- [A Survey on Negative Transfer](https://mlsanity.com/paper/2009.00909/) (2020)
- [A Tutorial on Meta-Reinforcement Learning](https://mlsanity.com/paper/2301.08028/) (2023)
- [Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2005.06800/) (2020)
- [Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts](https://mlsanity.com/paper/2311.11385/) (2023)
- [PaCo: Parameter-Compositional Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2210.11653/) (2022)
- [AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents](https://mlsanity.com/paper/2310.09971/) (2023)
- [Provable Benefits of Representational Transfer in Reinforcement Learning](https://mlsanity.com/paper/2205.14571/) (2022)
- [Provable Benefit of Multitask Representation Learning in Reinforcement Learning](https://mlsanity.com/paper/2206.05900/) (2022)
- [Misspecification in Inverse Reinforcement Learning](https://mlsanity.com/paper/2212.03201/) (2022)
- [Guided Meta-Policy Search](https://mlsanity.com/paper/1904.00956/) (2019)
- [Theoretical Analysis of Meta Reinforcement Learning: Generalization Bounds and Convergence Guarantees](https://mlsanity.com/paper/2405.13290/) (2024)
- [Meta-Reinforcement Learning via Language Instructions](https://mlsanity.com/paper/2209.04924/) (2022)
- [Contrastive Modules with Temporal Attention for Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2311.01075/) (2023)
- [An Optimal Tightness Bound for the Simulation Lemma](https://mlsanity.com/paper/2406.16249/) (2024)
- [Personalized Reinforcement Learning with a Budget of Policies](https://mlsanity.com/paper/2401.06514/) (2024)
- [Regret Bounds for Reinforcement Learning with Policy Advice](https://mlsanity.com/paper/1305.1027/) (2013)

## Built on by (most cited first, 4 of 4)

- [Structure Detection for Contextual Reinforcement Learning](https://mlsanity.com/paper/2601.08120/) (2026)
- [Adapting Actively on the Fly: Relevance-Guided Online Meta-Learning with Latent Concepts for Geospatial Discovery](https://mlsanity.com/paper/2602.17605/) (2026)
- [COSMOS: Model-Agnostic Personalized Federated Learning with Clustered Server Models and Pseudo-Label-Only Communication](https://mlsanity.com/paper/2605.11165/) (2026)
- [Task Specialization Fine-Tuning for Contextual Reinforcement Learning](https://mlsanity.com/paper/2608.17180/) (2026)

## Abstract (arXiv)

Many dynamic decision problems, such as robotic control, involve a series of tasks, many of which are unknown at training time. Typical approaches for these problems, such as multi-task and meta reinforcement learning, do not generalize well when the tasks are diverse. On the other hand, approaches that aim to tackle task diversity, such as using task embedding as policy context and task clustering, typically lack performance guarantees and require a large number of training tasks. To address these challenges, we propose a novel approach for learning a policy committee that includes at least one near-optimal policy with high probability for tasks encountered during execution. While we show that this problem is in general inapproximable, we present two practical algorithmic solutions. The first yields provable approximation and task sample complexity guarantees when tasks are low-dimensional (the best we can do due to inapproximability), whereas the second is a general and practical gradient-based approach. In addition, we provide a provable sample complexity bound for few-shot learning. Our experiments on MuJoCo and Meta-World show that the proposed approach outperforms state-of-the-art multi-task, meta-, and task clustering baselines in training, generalization, and few-shot learning, often by a large margin. Our code is available at https://github.com/CERL-WUSTL/PACMAN.