# DrEureka: Language Model Guided Sim-To-Real Transfer

arXiv: https://arxiv.org/abs/2406.01967 · HTML: https://mlsanity.com/paper/2406.01967/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- Builds on 56 of its 73 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Builds on (most cited first, 30 of 56)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [ProgPrompt: Generating Situated Robot Task Plans using Large Language Models](https://mlsanity.com/paper/2209.11302/) (2022)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Eureka: Human-Level Reward Design via Coding Large Language Models](https://mlsanity.com/paper/2310.12931/) (2023)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Language to Rewards for Robotic Skill Synthesis](https://mlsanity.com/paper/2306.08647/) (2023)
- [Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning](https://mlsanity.com/paper/1803.11347/) (2018)
- [Text2Motion: From Natural Language Instructions to Feasible Plans](https://mlsanity.com/paper/2303.12153/) (2023)
- [RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation](https://mlsanity.com/paper/2311.01455/) (2023)
- [Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience](https://mlsanity.com/paper/1810.05687/) (2018)
- [Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks](https://mlsanity.com/paper/1812.07252/) (2018)
- [Active Domain Randomization](https://mlsanity.com/paper/1904.04762/) (2019)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition](https://mlsanity.com/paper/2307.14535/) (2023)
- [DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality](https://mlsanity.com/paper/2210.13702/) (2022)
- [Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping](https://mlsanity.com/paper/1709.07857/) (2017)

## Built on by (most cited first, 30 of 83)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation](https://mlsanity.com/paper/2603.22435/) (2026)
- [ASPIRE: Agentic /Skills Discovery for Robotics](https://mlsanity.com/paper/2607.00272/) (2026)
- [Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments](https://mlsanity.com/paper/2409.05865/) (2024)
- [From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons](https://mlsanity.com/paper/2412.08442/) (2024)
- [ENPIRE: Agentic Robot Policy Self-Improvement in the Real World](https://mlsanity.com/paper/2606.19980/) (2026)
- [Safety Guardrails for LLM-Enabled Robots](https://mlsanity.com/paper/2503.07885/) (2025)
- [CurricuLLM: Automatic Task Curricula Design for Learning Complex Robot Skills using Large Language Models](https://mlsanity.com/paper/2409.18382/) (2024)
- [Playful Agentic Robot Learning](https://mlsanity.com/paper/2606.19419/) (2026)
- [RHO: Your Coding Agent is Secretly a Roboticist](https://mlsanity.com/paper/2606.16458/) (2026)
- [SPINE: Online Semantic Planning for Missions with Incomplete Natural Language Specifications in Unstructured Environments](https://mlsanity.com/paper/2410.03035/) (2024)
- [ORV: 4D Occupancy-centric Robot Video Generation](https://mlsanity.com/paper/2506.03079/) (2025)
- [Eurekaverse: Environment Curriculum Generation via Large Language Models](https://mlsanity.com/paper/2411.01775/) (2024)
- [A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI](https://mlsanity.com/paper/2505.01458/) (2025)
- [SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience](https://mlsanity.com/paper/2409.13678/) (2024)
- [Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments](https://mlsanity.com/paper/2504.19139/) (2025)
- [Evolution and The Knightian Blindspot of Machine Learning](https://mlsanity.com/paper/2501.13075/) (2025)
- [Air-Ground Collaboration for Language-Specified Missions in Unknown Environments](https://mlsanity.com/paper/2505.09108/) (2025)
- [FactorSim: Generative Simulation via Factorized Representation](https://mlsanity.com/paper/2409.17652/) (2024)
- [Balancing Act: Prioritization Strategies for LLM-Designed Restless Bandit Rewards](https://mlsanity.com/paper/2408.12112/) (2024)
- [GROVE: A Generalized Reward for Learning Open-Vocabulary Physical Skill](https://mlsanity.com/paper/2504.04191/) (2025)
- [Maestro: Orchestrating Robotics Modules with Vision-Language Models for Zero-Shot Generalist Robots](https://mlsanity.com/paper/2511.00917/) (2025)
- [Act-Observe-Rewrite: Multimodal Coding Agents as In-Context Policy Learners for Robot Manipulation](https://mlsanity.com/paper/2603.04466/) (2026)
- [RoboMorph: Evolving Robot Morphology using Large Language Models](https://mlsanity.com/paper/2407.08626/) (2024)
- [Helpful DoggyBot: Open-World Object Fetching using Legged Robots and Vision-Language Models](https://mlsanity.com/paper/2410.00231/) (2024)
- [Generative AI Agents in Autonomous Machines: A Safety Perspective](https://mlsanity.com/paper/2410.15489/) (2024)
- [ICPL: Few-shot In-context Preference Learning via LLMs](https://mlsanity.com/paper/2410.17233/) (2024)
- [Jailbreaking LLM-Controlled Robots](https://mlsanity.com/paper/2410.13691/) (2024)
- [The Duality of Generative AI and Reinforcement Learning in Robotics: A Review](https://mlsanity.com/paper/2410.16411/) (2024)
- [Al-Khwarizmi: Discovering Physical Laws with Foundation Models](https://mlsanity.com/paper/2502.01702/) (2025)

## Abstract (arXiv)

Transferring policies learned in simulation to the real world is a promising strategy for acquiring robot skills at scale. However, sim-to-real approaches typically rely on manual design and tuning of the task reward function as well as the simulation physics parameters, rendering the process slow and human-labor intensive. In this paper, we investigate using Large Language Models (LLMs) to automate and accelerate sim-to-real design. Our LLM-guided sim-to-real approach, DrEureka, requires only the physics simulation for the target task and automatically constructs suitable reward functions and domain randomization distributions to support real-world transfer. We first demonstrate that our approach can discover sim-to-real configurations that are competitive with existing human-designed ones on quadruped locomotion and dexterous manipulation tasks. Then, we showcase that our approach is capable of solving novel robot tasks, such as quadruped balancing and walking atop a yoga ball, without iterative manual design.