# Learning to Design and Use Tools for Robotic Manipulation

arXiv: https://arxiv.org/abs/2311.00754 · HTML: https://mlsanity.com/paper/2311.00754/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 40 references that are in the map.
- First posted on arXiv 1 Nov 2023.

## Builds on (most cited first, 26 of 26)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Embodied Intelligence via Learning and Evolution](https://mlsanity.com/paper/2102.02202/) (2021)
- [Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity](https://mlsanity.com/paper/1902.05546/) (2019)
- [KETO: Learning Keypoint Representations for Tool Manipulation](https://mlsanity.com/paper/1910.11977/) (2019)
- [Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning](https://mlsanity.com/paper/1907.09620/) (2019)
- [Reinforcement Learning for Improving Agent Design](https://mlsanity.com/paper/1810.03779/) (2018)
- [Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning](https://mlsanity.com/paper/1801.01432/) (2018)
- [Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision](https://mlsanity.com/paper/1806.09266/) (2018)
- [Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots](https://mlsanity.com/paper/2201.09863/) (2022)
- [DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools](https://mlsanity.com/paper/2203.17275/) (2022)
- [Deep Affordance Foresight: Planning Through What Can Be Done in the Future](https://mlsanity.com/paper/2011.08424/) (2020)
- [Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design](https://mlsanity.com/paper/2110.03659/) (2021)
- [Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning](https://mlsanity.com/paper/1911.06832/) (2019)
- [Physical Design using Differentiable Learned Simulators](https://mlsanity.com/paper/2202.00728/) (2022)
- [Improvisation through Physical Understanding: Using Novel Objects as Tools with Visual Foresight](https://mlsanity.com/paper/1904.05538/) (2019)
- [Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.04460/) (2020)
- [Data-efficient Learning of Morphology and Controller for a Microrobot](https://mlsanity.com/paper/1905.01334/) (2019)
- [Task-Agnostic Morphology Evolution](https://mlsanity.com/paper/2102.13100/) (2021)
- [Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learning](https://mlsanity.com/paper/2203.13733/) (2022)
- [Fit2Form: 3D Generative Model for Robot Gripper Form Design](https://mlsanity.com/paper/2011.06498/) (2020)
- [Forward Prediction for Physical Reasoning](https://mlsanity.com/paper/2006.10734/) (2020)
- [Learning Tool Morphology for Contact-Rich Manipulation Tasks with Differentiable Simulation](https://mlsanity.com/paper/2211.02201/) (2022)
- [Learning to Design and Construct Bridge without Blueprint](https://mlsanity.com/paper/2108.02439/) (2021)
- [Task-Specific Design Optimization and Fabrication for Inflated-Beam Soft Robots with Growable Discrete Joints](https://mlsanity.com/paper/2103.04942/) (2021)
- [Tool as Embodiment for Recursive Manipulation](https://mlsanity.com/paper/2112.00359/) (2021)
- [Tool Macgyvering: A Novel Framework for Combining Tool Substitution and Construction](https://mlsanity.com/paper/2008.10638/) (2020)

## Built on by (most cited first, 7 of 7)

- [VLMgineer: Vision Language Models as Robotic Toolsmiths](https://mlsanity.com/paper/2507.12644/) (2025)
- [House of Dextra: Cross-embodied Co-design for Dexterous Hands](https://mlsanity.com/paper/2512.03743/) (2025)
- [RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skills](https://mlsanity.com/paper/2506.14763/) (2025)
- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [Designing Tools with Control Confidence](https://mlsanity.com/paper/2510.12630/) (2025)
- [Latent Diffeomorphic Co-Design of End-Effectors for Deformable and Fragile Object Manipulation](https://mlsanity.com/paper/2602.17921/) (2026)
- [Robot Metacognition: Decision Making with Confidence for Tool Invention](https://mlsanity.com/paper/2511.16390/) (2025)

## Abstract (arXiv)

When limited by their own morphologies, humans and some species of animals have the remarkable ability to use objects from the environment toward accomplishing otherwise impossible tasks. Robots might similarly unlock a range of additional capabilities through tool use. Recent techniques for jointly optimizing morphology and control via deep learning are effective at designing locomotion agents. But while outputting a single morphology makes sense for locomotion, manipulation involves a variety of strategies depending on the task goals at hand. A manipulation agent must be capable of rapidly prototyping specialized tools for different goals. Therefore, we propose learning a designer policy, rather than a single design. A designer policy is conditioned on task information and outputs a tool design that helps solve the task. A design-conditioned controller policy can then perform manipulation using these tools. In this work, we take a step towards this goal by introducing a reinforcement learning framework for jointly learning these policies. Through simulated manipulation tasks, we show that this framework is more sample efficient than prior methods in multi-goal or multi-variant settings, can perform zero-shot interpolation or fine-tuning to tackle previously unseen goals, and allows tradeoffs between the complexity of design and control policies under practical constraints. Finally, we deploy our learned policies onto a real robot. Please see our supplementary video and website at https://robotic-tool-design.github.io/ for visualizations.