mlsanityData to 7 Oct 2026

One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control

arXiv 2007.04976 ↗First posted 9 Jul 2020cs.LG cs.CV stat.MLcited by 114

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning is typically concerned with learning control policies tailored to a particular agent. We investigate whether there exists a single global policy that can generalize to control a wide variety of agent morphologies -- ones in which even dimensionality of state and action spaces changes. We propose to express this global policy as a collection of identical modular neural networks, dubbed as Shared Modular Policies (SMP), that correspond to each of the agent's actuators. Every module is only responsible for controlling its corresponding actuator and receives information from only its local sensors. In addition, messages are passed between modules, propagating information between distant modules. We show that a single modular policy can successfully generate locomotion behaviors for several planar agents with different skeletal structures such as monopod hoppers, quadrupeds, bipeds, and generalize to variants not seen during training -- a process that would normally require training and manual hyperparameter tuning for each morphology. We observe that a wide variety of drastically diverse locomotion styles across morphologies as well as centralized coordination emerges via message passing between decentralized modules purely from the reinforcement learning objective. Videos and code at https://huangwl18.github.io/modular-rl/

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 26; 2023: 17; 2024: 23; 2025: 23; 2026: 11.

114 papers.

Sort:

By area:

  1. 2022A Generalist Agentcited by 802
  2. 2022Multi-Game Decision Transformerscited by 212
  3. 2022GNM: A General Navigation Model to Drive Any Robotcited by 161
  4. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
  5. 2021Embodied Intelligence via Learning and Evolutioncited by 154
  6. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 133
  7. 2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108
  8. 2022MetaMorph: Learning Universal Controllers with Transformerscited by 78
  9. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 65
  10. 2025Humanoid Policy ~ Human Policycited by 65
  11. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learningcited by 56
  12. 2025FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policiescited by 56
38 more
  1. 2022GenLoco: Generalized Locomotion Controllers for Quadrupedal Robotscited by 36
  2. 2021Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Designcited by 34
  3. 2024One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotioncited by 34
  4. 2021Accelerating Quadratic Optimization with Reinforcement Learningcited by 30
  5. 2022Challenges and Opportunities in Deep Reinforcement Learning with Graph Neural Networks: A Comprehensive review of Algorithms and Applicationscited by 27
  6. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  7. 2024Body Transformer: Leveraging Robot Embodiment for Policy Learningcited by 26
  8. 2022AnyMorph: Learning Transferable Polices By Inferring Agent Morphologycited by 25
  9. 2023A Generalist Dynamics Model for Controlcited by 24
  10. 2021Task-Agnostic Morphology Evolutioncited by 22
  11. 2021Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platformscited by 22
  12. 2022Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learningcited by 21
  13. 2023Universal Morphology Control via Contextual Modulationcited by 21
  14. 2023Graph Decision Transformercited by 20
  15. 2021ProTo: Program-Guided Transformer for Program-Guided Taskscited by 19
  16. 2021Learning Modular Robot Control Policiescited by 18
  17. 2021In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specificationscited by 17
  18. 2026LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfercited by 17
  19. 2022GLSO: Grammar-guided Latent Space Optimization for Sample-efficient Robot Design Automationcited by 16
  20. 2022Low-Rank Modular Reinforcement Learning via Muscle Synergycited by 16
  21. 2023ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robotscited by 15
  22. 2022Multi-Agent Policy Transfer via Task Relationship Modelingcited by 14
  23. 2022QGNN: Value Function Factorisation with Graph Neural Networkscited by 14
  24. 2025LocoFormer: Generalist Locomotion via Long-context Adaptationcited by 14
  25. 2023Symmetry-Aware Robot Design with Structured Subgroupscited by 13
  26. 2022REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfercited by 12
  27. 2022A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based Policy Learningcited by 12
  28. 2022A System for Morphology-Task Generalization via Unified Representation and Behavior Distillationcited by 12
  29. 2025Towards Embodiment Scaling Laws in Robot Locomotioncited by 12
  30. 2021Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximizationcited by 11
  31. 2023Transformers in Reinforcement Learning: A Surveycited by 11
  32. 2023Efficient RL via Disentangled Environment and Agent Representationscited by 11
  33. 2024Artificial Intelligence for Operations Research: Revolutionizing the Operations Research Processcited by 11
  34. 2024Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Controlcited by 11
  35. 2024Entity-based Reinforcement Learning for Autonomous Cyber Defencecited by 11
  36. 2023Subequivariant Graph Reinforcement Learning in 3D Environmentscited by 10
  37. 2025Geometry-aware RL for Manipulation of Varying Shapes and Deformable Objectscited by 10
  38. 2025OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learningcited by 10

50 most cited of 114 All 114 (JSON)