# An End-to-End Differentiable Framework for Contact-Aware Robot Design

arXiv: https://arxiv.org/abs/2107.07501 · HTML: https://mlsanity.com/paper/2107.07501/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 55 references that are in the map.
- First posted on arXiv 15 Jul 2021.

## Builds on (most cited first, 24 of 24)

- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids](https://mlsanity.com/paper/1810.01566/) (2018)
- [Deep Dynamics Models for Learning Dexterous Manipulation](https://mlsanity.com/paper/1909.11652/) (2019)
- [Embodied Intelligence via Learning and Evolution](https://mlsanity.com/paper/2102.02202/) (2021)
- [A Compositional Object-Based Approach to Learning Physical Dynamics](https://mlsanity.com/paper/1612.00341/) (2016)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics](https://mlsanity.com/paper/2104.03311/) (2021)
- [gradSim: Differentiable simulation for system identification and visuomotor control](https://mlsanity.com/paper/2104.02646/) (2021)
- [ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics](https://mlsanity.com/paper/1810.01054/) (2018)
- [Flexible Neural Representation for Physics Prediction](https://mlsanity.com/paper/1806.08047/) (2018)
- [Scalable Differentiable Physics for Learning and Control](https://mlsanity.com/paper/2007.02168/) (2020)
- [Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning](https://mlsanity.com/paper/1801.01432/) (2018)
- [Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning](https://mlsanity.com/paper/1911.06832/) (2019)
- [Neural Graph Evolution: Towards Efficient Automatic Robot Design](https://mlsanity.com/paper/1906.05370/) (2019)
- [Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.04460/) (2020)
- [Fit2Form: 3D Generative Model for Robot Gripper Form Design](https://mlsanity.com/paper/2011.06498/) (2020)
- [Multi-Objective Graph Heuristic Search for Terrestrial Robot Design](https://mlsanity.com/paper/2107.05858/) (2021)
- [Interactive Differentiable Simulation](https://mlsanity.com/paper/1905.10706/) (2019)
- [Functional Co-Optimization of Articulated Robots](https://mlsanity.com/paper/1707.06617/) (2017)
- [Emergent Hand Morphology and Control from Optimizing Robust Grasps of Diverse Objects](https://mlsanity.com/paper/2012.12209/) (2020)

## Built on by (most cited first, 30 of 54)

- [Accelerated Policy Learning with Parallel Differentiable Simulation](https://mlsanity.com/paper/2204.07137/) (2022)
- [Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes](https://mlsanity.com/paper/2211.11744/) (2022)
- [FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation](https://mlsanity.com/paper/2303.02346/) (2023)
- [Physical Design using Differentiable Learned Simulators](https://mlsanity.com/paper/2202.00728/) (2022)
- [RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation](https://mlsanity.com/paper/2205.05678/) (2022)
- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Adaptive Horizon Actor-Critic for Policy Learning in Contact-Rich Differentiable Simulation](https://mlsanity.com/paper/2405.17784/) (2024)
- [Dress-1-to-3: Single Image to Simulation-Ready 3D Outfit with Diffusion Prior and Differentiable Physics](https://mlsanity.com/paper/2502.03449/) (2025)
- [ASAP: Automated Sequence Planning for Complex Robotic Assembly with Physical Feasibility](https://mlsanity.com/paper/2309.16909/) (2023)
- [QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfer](https://mlsanity.com/paper/2404.07988/) (2024)
- [VLMgineer: Vision Language Models as Robotic Toolsmiths](https://mlsanity.com/paper/2507.12644/) (2025)
- [Learning Tool Morphology for Contact-Rich Manipulation Tasks with Differentiable Simulation](https://mlsanity.com/paper/2211.02201/) (2022)
- [Dynamics-Guided Diffusion Model for Sensor-less Robot Manipulator Design](https://mlsanity.com/paper/2402.15038/) (2024)
- [MORPH: Design Co-optimization with Reinforcement Learning via a Differentiable Hardware Model Proxy](https://mlsanity.com/paper/2309.17227/) (2023)
- [Text2Robot: Evolutionary Robot Design from Text Descriptions](https://mlsanity.com/paper/2406.19963/) (2024)
- [Neural Robot Dynamics](https://mlsanity.com/paper/2508.15755/) (2025)
- [Differentiable Physics Simulations with Contacts: Do They Have Correct Gradients w.r.t. Position, Velocity and Control?](https://mlsanity.com/paper/2207.05060/) (2022)
- [Model-Based Reparameterization Policy Gradient Methods: Theory and Practical Algorithms](https://mlsanity.com/paper/2310.19927/) (2023)
- [CoReVLA: A Dual-Stage End-to-End Autonomous Driving Framework for Long-Tail Scenarios via Collect-and-Refine](https://mlsanity.com/paper/2509.15968/) (2025)
- [One-Shot Real-to-Sim via End-to-End Differentiable Simulation and Rendering](https://mlsanity.com/paper/2412.00259/) (2024)
- [Local object crop collision network for efficient simulation of non-convex objects in GPU-based simulators](https://mlsanity.com/paper/2304.09439/) (2023)
- [Improving Gradient Computation for Differentiable Physics Simulation with Contacts](https://mlsanity.com/paper/2305.00092/) (2023)
- [JAX-MPM: A Learning-Augmented Differentiable Meshfree Framework for GPU-Accelerated Lagrangian Simulation and Geophysical Inverse Modeling](https://mlsanity.com/paper/2507.04192/) (2025)
- [House of Dextra: Cross-embodied Co-design for Dexterous Hands](https://mlsanity.com/paper/2512.03743/) (2025)
- [Tactile-Filter: Interactive Tactile Perception for Part Mating](https://mlsanity.com/paper/2303.06034/) (2023)
- [STAMP: Differentiable Task and Motion Planning via Stein Variational Gradient Descent](https://mlsanity.com/paper/2310.01775/) (2023)
- [Task2Morph: Differentiable Task-inspired Framework for Contact-Aware Robot Design](https://mlsanity.com/paper/2403.19093/) (2024)
- [TopoGaussian: Inferring Internal Topology Structures from Visual Clues](https://mlsanity.com/paper/2503.12343/) (2025)
- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [Simulation-Ready Cluttered Scene Estimation via Physics-aware Joint Shape and Pose Optimization](https://mlsanity.com/paper/2602.20150/) (2026)

## Abstract (arXiv)

The current dominant paradigm for robotic manipulation involves two separate stages: manipulator design and control. Because the robot's morphology and how it can be controlled are intimately linked, joint optimization of design and control can significantly improve performance. Existing methods for co-optimization are limited and fail to explore a rich space of designs. The primary reason is the trade-off between the complexity of designs that is necessary for contact-rich tasks against the practical constraints of manufacturing, optimization, contact handling, etc. We overcome several of these challenges by building an end-to-end differentiable framework for contact-aware robot design. The two key components of this framework are: a novel deformation-based parameterization that allows for the design of articulated rigid robots with arbitrary, complex geometry, and a differentiable rigid body simulator that can handle contact-rich scenarios and computes analytical gradients for a full spectrum of kinematic and dynamic parameters. On multiple manipulation tasks, our framework outperforms existing methods that either only optimize for control or for design using alternate representations or co-optimize using gradient-free methods.