# On Assessing the Usefulness of Proxy Domains for Developing and Evaluating Embodied Agents

arXiv: https://arxiv.org/abs/2109.14516 · HTML: https://mlsanity.com/paper/2109.14516/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 29 Sep 2021.

## Builds on (most cited first, 26 of 26)

- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks](https://mlsanity.com/paper/1612.05424/) (2016)
- [Learning from Simulated and Unsupervised Images through Adversarial Training](https://mlsanity.com/paper/1612.07828/) (2016)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [One-Shot Visual Imitation Learning via Meta-Learning](https://mlsanity.com/paper/1709.04905/) (2017)
- [Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience](https://mlsanity.com/paper/1810.05687/) (2018)
- [Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks](https://mlsanity.com/paper/1812.07252/) (2018)
- [Active Domain Randomization](https://mlsanity.com/paper/1904.04762/) (2019)
- [One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning](https://mlsanity.com/paper/1802.01557/) (2018)
- [Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping](https://mlsanity.com/paper/1709.07857/) (2017)
- [gradSim: Differentiable simulation for system identification and visuomotor control](https://mlsanity.com/paper/2104.02646/) (2021)
- [Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model](https://mlsanity.com/paper/1610.03518/) (2016)
- [Policy Transfer with Strategy Optimization](https://mlsanity.com/paper/1810.05751/) (2018)
- [Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance?](https://mlsanity.com/paper/1912.06321/) (2019)
- [Adapting Deep Visuomotor Representations with Weak Pairwise Constraints](https://mlsanity.com/paper/1511.07111/) (2015)
- [Learning to Augment Synthetic Images for Sim2Real Policy Transfer](https://mlsanity.com/paper/1903.07740/) (2019)
- [Adversarial Discriminative Sim-to-real Transfer of Visuo-motor Policies](https://mlsanity.com/paper/1709.05746/) (2017)
- [A Data-Efficient Framework for Training and Sim-to-Real Transfer of Navigation Policies](https://mlsanity.com/paper/1810.04871/) (2018)
- [Experience Selection Using Dynamics Similarity for Efficient Multi-Source Transfer Learning Between Robots](https://mlsanity.com/paper/2003.13150/) (2020)
- [The AI Driving Olympics at NeurIPS 2018](https://mlsanity.com/paper/1903.02503/) (2019)
- [Traversing the Reality Gap via Simulator Tuning](https://mlsanity.com/paper/2003.01369/) (2020)
- [Integrated Benchmarking and Design for Reproducible and Accessible Evaluation of Robotic Agents](https://mlsanity.com/paper/2009.04362/) (2020)
- [Crossing the Reality Gap: a Short Introduction to the Transferability Approach](https://mlsanity.com/paper/1307.1870/) (2013)

## Abstract (arXiv)

In many situations it is either impossible or impractical to develop and evaluate agents entirely on the target domain on which they will be deployed. This is particularly true in robotics, where doing experiments on hardware is much more arduous than in simulation. This has become arguably more so in the case of learning-based agents. To this end, considerable recent effort has been devoted to developing increasingly realistic and higher fidelity simulators. However, we lack any principled way to evaluate how good a "proxy domain" is, specifically in terms of how useful it is in helping us achieve our end objective of building an agent that performs well in the target domain. In this work, we investigate methods to address this need. We begin by clearly separating two uses of proxy domains that are often conflated: 1) their ability to be a faithful predictor of agent performance and 2) their ability to be a useful tool for learning. In this paper, we attempt to clarify the role of proxy domains and establish new proxy usefulness (PU) metrics to compare the usefulness of different proxy domains. We propose the relative predictive PU to assess the predictive ability of a proxy domain and the learning PU to quantify the usefulness of a proxy as a tool to generate learning data. Furthermore, we argue that the value of a proxy is conditioned on the task that it is being used to help solve. We demonstrate how these new metrics can be used to optimize parameters of the proxy domain for which obtaining ground truth via system identification is not trivial.