mlsanityData to 14 Oct 2026

Policy Search, Retrieval, and Composition via Task Similarity in Collaborative Agentic Systems

arXiv 2506.05577 ↗First posted 5 Jun 2025cs.LG cs.AI cs.MAcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Agentic AI aims to create systems that set their own goals, adapt proactively to change, and refine behavior through continuous experience. Recent advances suggest that, when facing multiple and unforeseen tasks, agents could benefit from sharing machine-learned knowledge and reusing policies that have already been fully or partially learned by other agents. However, how to query, select, and retrieve policies from a pool of agents, and how to integrate such policies remains a largely unexplored area. This study explores how an agent decides what knowledge to select, from whom, and when and how to integrate it in its own policy in order to accelerate its own learning. The proposed algorithm, Modular Sharing and Composition in Collective Learning (MOSAIC), improves learning in agentic collectives by combining (1) knowledge selection using performance signals and cosine similarity on Wasserstein task embeddings, (2) modular and transferable neural representations via masks, and (3) policy integration, composition and fine-tuning. MOSAIC outperforms isolated learners and global sharing approaches in both learning speed and overall performance, and in some cases solves tasks that isolated agents cannot. The results also demonstrate that selective, goal-driven reuse leads to less susceptibility to task interference. We also observe the emergence of self-organization, where agents solving simpler tasks accelerate the learning of harder ones through shared knowledge.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 89 of its 128 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

89 of 128 references.

References · 89

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  6. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  7. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  8. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  9. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  10. 2016Concrete Problems in AI Safetycited by 1,642
  11. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  12. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
38 more
  1. 2016Progressive Neural Networkscited by 1,361
  2. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  3. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  4. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  5. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  6. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  7. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  8. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  9. 2017Deep Reinforcement Learning that Matterscited by 754
  10. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
  11. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  12. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  13. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 505
  14. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  15. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  16. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  17. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
  18. 2015Neural Module Networkscited by 376
  19. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
  20. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
  21. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
  22. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  23. 2019Continual learning with hypernetworkscited by 273
  24. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  25. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  26. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 243
  27. 2018Selective Experience Replay for Lifelong Learningcited by 241
  28. 2019What's Hidden in a Randomly Weighted Neural Network?cited by 237
  29. 2022Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpointscited by 209
  30. 2021Reinforcement Learning with Prototypical Representationscited by 179
  31. 2018SpotTune: Transfer Learning through Adaptive Fine-tuningcited by 168
  32. 2025Multi-Agent Risks from Advanced AIcited by 163
  33. 2020Federated Continual Learning with Weighted Inter-client Transfercited by 161
  34. 2024Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLMcited by 109
  35. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 97
  36. 2024Multi-LoRA Composition for Image Generationcited by 80
  37. 2024Reinforcement Learning Enhanced LLMs: A Surveycited by 71
  38. 2021Continual Learning via Local Module Compositioncited by 63

50 most cited of 89 All 89 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026How to Find and Reuse Policies for Continuous Adaptation in Lifelong Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Modular Reinforcement Learning For Cooperative Swarms
  2. 2026MOSAIC: A Universal Agent-Level Interface for Cross-Paradigm Agent Mixing and Human-AI Collaboration
  3. 2025Multi-agent In-context Coordination via Decentralized Memory Retrieval
  4. 2025From Solo to Symphony: Orchestrating Multi-Agent Collaboration with Single-Agent Demos
  5. 2026Evaluating XAI Support From A Hierarchical Reinforcement Learning Policy in Human-Agent Collaboration
  6. 2026Learning to Share: Selective Memory for Efficient Parallel Agentic Systems
  7. 2026MATES: Learning Multi-Agent Interactions by Transforming Observations for Frozen Single-Agent Policies ↗
  8. 2022Modular Adaptive Policy Selection for Multi-Task Imitation Learning through Task Division