mlsanityData to 7 Oct 2026

Reinforcement Learning for Improving Agent Design

arXiv 1810.03779 ↗First posted 9 Oct 2018cs.LG stat.MLcited by 55

Not on the map. Why?

Abstract (arXiv)

In many reinforcement learning tasks, the goal is to learn a policy to manipulate an agent, whose design is fixed, to maximize some notion of cumulative reward. The design of the agent's physical structure is rarely optimized for the task at hand. In this work, we explore the possibility of learning a version of the agent's design that is better suited for its task, jointly with the policy. We propose an alteration to the popular OpenAI Gym framework, where we parameterize parts of an environment, and allow an agent to jointly learn to modify these environment parameters along with its policy. We demonstrate that an agent can learn a better structure of its body that is not only better suited for the task, but also facilitates policy learning. Joint learning of policy and structure may even uncover design principles that are useful for assisted-design applications. Videos of results at https://designrl.github.io/

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 8; 2022: 17; 2023: 9; 2024: 3; 2025: 9; 2026: 8.

55 papers.

Sort:

By area:

  1. 2021Embodied Intelligence via Learning and Evolutioncited by 154
  2. 2025Reward Shaping to Mitigate Reward Hacking in RLHFcited by 88
  3. 2022MetaMorph: Learning Universal Controllers with Transformerscited by 78
  4. 2022Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robotscited by 44
  5. 2022Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimalitycited by 35
  6. 2021Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Designcited by 34
  7. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  8. 2021Task-Agnostic Morphology Evolutioncited by 22
  9. 2022Combining Evolution and Deep Reinforcement Learning for Policy Search: a Surveycited by 20
  10. 2021TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RLcited by 18
  11. 2023SoftZoo: A Soft Robot Co-design Benchmark For Locomotion In Diverse Environmentscited by 18
  12. 2023Searching for High-Value Molecules Using Reinforcement Learning and Transformerscited by 18
38 more
  1. 2023Symmetry-Aware Robot Design with Structured Subgroupscited by 13
  2. 2022REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfercited by 12
  3. 2022Sample Complexity of Nonparametric Off-Policy Evaluation on Low-Dimensional Manifolds using Deep Networkscited by 12
  4. 2022From Universal Humanoid Control to Automatic Physically Valid Character Creationcited by 11
  5. 2023Reinforcement Learning for Generative AI: A Surveycited by 11
  6. 2021DiffAqua: A Differentiable Computational Design Pipeline for Soft Underwater Swimmers with Shape Interpolationcited by 9
  7. 2022Meta Reinforcement Learning for Optimal Design of Legged Robotscited by 9
  8. 2025BodyGen: Advancing Towards Efficient Embodiment Co-Designcited by 9
  9. 2025VLMgineer: Vision Language Models as Robotic Toolsmithscited by 9
  10. 2020Off-Policy Adversarial Inverse Reinforcement Learningcited by 8
  11. 2023EvoTorch: Scalable Evolutionary Computation in Pythoncited by 8
  12. 2023MORPH: Design Co-optimization with Reinforcement Learning via a Differentiable Hardware Model Proxycited by 8
  13. 2023Reinforcement learning for freeform robot designcited by 7
  14. 2023Learning to Design and Use Tools for Robotic Manipulationcited by 7
  15. 2024DittoGym: Learning to Control Soft Shape-Shifting Robotscited by 6
  16. 2021Efficient Hyperparameter Optimization for Physics-based Character Animationcited by 4
  17. 2022Creative Problem Solving in Artificially Intelligent Agents: A Survey and Frameworkcited by 4
  18. 2025RoboMoRe: LLM-based Robot Co-design via Joint Optimization of Morphology and Rewardcited by 4
  19. 2025Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challengescited by 4
  20. 2021Carle's Game: An Open-Ended Challenge in Exploratory Machine Creativitycited by 3
  21. 2022Biological Robots: Perspectives on an Emerging Interdisciplinary Fieldcited by 3
  22. 2022Co-Imitation: Learning Design and Behaviour by Imitationcited by 3
  23. 2021What Robot do I Need? Fast Co-Adaptation of Morphology and Control using Graph Neural Networkscited by 2
  24. 2022The Introspective Agent: Interdependence of Strategy, Physiology, and Sensing for Embodied Agentscited by 2
  25. 2022Adversarial Body Shape Search for Legged Robotscited by 2
  26. 2023ECoDe: A Sample-Efficient Method for Co-Design of Robotic Agentscited by 2
  27. 2024CompetEvo: Towards Morphological Evolution from Competitioncited by 2
  28. 2024Reinforcement Learning for Efficient Design and Control Co-optimisation of Energy Systemscited by 2
  29. 2025Exploring Incremental Unlearning: Techniques, Challenges, and Future Directionscited by 2
  30. 2025A Design Co-Pilot for Task-Tailored Manipulatorscited by 2
  31. 2022Co-design of Embodied Neural Intelligence via Constrained Evolutioncited by 1
  32. 2025The Emergence of Complex Behavior in Large-Scale Ecological Environmentscited by 1
  33. 2025Mathematical Framing for Different Agent Strategiescited by 1
  34. 2026Latent Diffeomorphic Co-Design of End-Effectors for Deformable and Fragile Object Manipulationcited by 1
  35. 2026Evolving Embodied Intelligence: Graph Neural Network--Driven Co-Design of Morphology and Control in Soft Roboticscited by 1
  36. 2026Shape Your Body: Value Gradients for Multi-Embodiment Robot Designcited by 1
  37. 2022Reinforcement Learning Agent Design and Optimization with Bandwidth Allocation Modelcited by 0
  38. 2022Selecting Mechanical Parameters of a Monopode Jumping System with Reinforcement Learningcited by 0

50 most cited of 55 All 55 (JSON)