# AnyMorph: Learning Transferable Polices By Inferring Agent Morphology

arXiv: https://arxiv.org/abs/2206.12279 · HTML: https://mlsanity.com/paper/2206.12279/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 17 Jun 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Offline Reinforcement Learning as One Big Sequence Modeling Problem](https://mlsanity.com/paper/2106.02039/) (2021)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [Graph networks as learnable physics engines for inference and control](https://mlsanity.com/paper/1806.01242/) (2018)
- [Analyzing the Structure of Attention in a Transformer Language Model](https://mlsanity.com/paper/1906.04284/) (2019)
- [Recurrent Independent Mechanisms](https://mlsanity.com/paper/1909.10893/) (2019)
- [Simple but Effective: CLIP Embeddings for Embodied AI](https://mlsanity.com/paper/2111.09888/) (2021)
- [Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning](https://mlsanity.com/paper/1511.06342/) (2015)
- [Planning with Goal-Conditioned Policies](https://mlsanity.com/paper/1911.08453/) (2019)
- [Language as an Abstraction for Hierarchical Deep Reinforcement Learning](https://mlsanity.com/paper/1906.07343/) (2019)
- [Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfer](https://mlsanity.com/paper/1609.07088/) (2016)
- [One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control](https://mlsanity.com/paper/2007.04976/) (2020)
- [My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Control](https://mlsanity.com/paper/2010.01856/) (2020)
- [Hierarchically Decoupled Imitation for Morphological Transfer](https://mlsanity.com/paper/2003.01709/) (2020)
- [Functional Regularization for Reinforcement Learning via Learned Fourier Features](https://mlsanity.com/paper/2112.03257/) (2021)
- [Modularity in Reinforcement Learning via Algorithmic Independence in Credit Assignment](https://mlsanity.com/paper/2106.14993/) (2021)

## Built on by (most cited first, 27 of 27)

- [One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotion](https://mlsanity.com/paper/2409.06366/) (2024)
- [On Transforming Reinforcement Learning by Transformer: The Development Trajectory](https://mlsanity.com/paper/2212.14164/) (2022)
- [Universal Morphology Control via Contextual Modulation](https://mlsanity.com/paper/2302.11070/) (2023)
- [ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robots](https://mlsanity.com/paper/2310.10486/) (2023)
- [Towards Embodiment Scaling Laws in Robot Locomotion](https://mlsanity.com/paper/2505.05753/) (2025)
- [A System for Morphology-Task Generalization via Unified Representation and Behavior Distillation](https://mlsanity.com/paper/2211.14296/) (2022)
- [Subequivariant Graph Reinforcement Learning in 3D Environments](https://mlsanity.com/paper/2305.18951/) (2023)
- [Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Control](https://mlsanity.com/paper/2402.06570/) (2024)
- [Geometry-aware RL for Manipulation of Varying Shapes and Deformable Objects](https://mlsanity.com/paper/2502.07005/) (2025)
- [BodyGen: Advancing Towards Efficient Embodiment Co-Design](https://mlsanity.com/paper/2503.00533/) (2025)
- [DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Body](https://mlsanity.com/paper/2209.14218/) (2022)
- [The Duality of Generative AI and Reinforcement Learning in Robotics: A Review](https://mlsanity.com/paper/2410.16411/) (2024)
- [Curriculum Reinforcement Learning via Morphology-Environment Co-Evolution](https://mlsanity.com/paper/2309.12529/) (2023)
- [Cross Domain Policy Transfer with Effect Cycle-Consistency](https://mlsanity.com/paper/2403.02018/) (2024)
- [A Behavior-Aware Approach for Deep Reinforcement Learning in Non-stationary Environments without Known Change Points](https://mlsanity.com/paper/2405.14214/) (2024)
- [Feudal Graph Reinforcement Learning](https://mlsanity.com/paper/2304.05099/) (2023)
- [HeteroMorpheus: Universal Control Based on Morphological Heterogeneity Modeling](https://mlsanity.com/paper/2408.01230/) (2024)
- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [Morphology-Conditioned World Model for Cross-Embodiment Quadrupedal Locomotion](https://mlsanity.com/paper/2604.08780/) (2026)
- [Meta-Evolve: Continuous Robot Evolution for One-to-many Policy Transfer](https://mlsanity.com/paper/2405.03534/) (2024)
- [Arnold: A multi-task, multi-embodiment muscle transformer policy](https://mlsanity.com/paper/2508.18066/) (2025)
- [MetaLoco: Universal Quadrupedal Locomotion with Meta-Reinforcement Learning and Motion Imitation](https://mlsanity.com/paper/2407.17502/) (2024)
- [Knowledge Diversion for Efficient Morphology Control and Policy Transfer](https://mlsanity.com/paper/2512.09796/) (2025)
- [Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning](https://mlsanity.com/paper/2607.16534/) (2026)
- [Efficient Morphology-Aware Policy Transfer to New Embodiments](https://mlsanity.com/paper/2508.03660/) (2025)
- [Forecasting in Offline Reinforcement Learning for Non-stationary Environments](https://mlsanity.com/paper/2512.01987/) (2025)
- [MorFiC: Fixing Value Miscalibration for Zero-Shot Quadruped Transfer](https://mlsanity.com/paper/2603.14554/) (2026)

## Abstract (arXiv)

The prototypical approach to reinforcement learning involves training policies tailored to a particular agent from scratch for every new morphology. Recent work aims to eliminate the re-training of policies by investigating whether a morphology-agnostic policy, trained on a diverse set of agents with similar task objectives, can be transferred to new agents with unseen morphologies without re-training. This is a challenging problem that required previous approaches to use hand-designed descriptions of the new agent's morphology. Instead of hand-designing this description, we propose a data-driven method that learns a representation of morphology directly from the reinforcement learning objective. Ours is the first reinforcement learning algorithm that can train a policy to generalize to new agent morphologies without requiring a description of the agent's morphology in advance. We evaluate our approach on the standard benchmark for agent-agnostic control, and improve over the current state of the art in zero-shot generalization to new agents. Importantly, our method attains good performance without an explicit description of morphology.