mlsanityData to 7 Oct 2026

RHO: Your Coding Agent is Secretly a Roboticist

arXiv 2606.16458 ↗First posted 15 Jun 2026cs.ROcited by 18

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Code-as-Policies (CaP) has shown that large language models (LLMs) can write code to solve robotics tasks by composing perception, planning, and control primitives. Recent CaP systems, however, rely on multi-turn code-generation loops at test time, which is often infeasible for real-time robot control. We introduce Robotics Harness Optimization (RHO), a novel paradigm in which tool-enabled coding agents, at training time, propose and search for interpretable, neurosymbolic multi-file policy repositories (Repositories-as-Policies) that compose these primitives rather than a single prompt, function, or file. RHO searches with reflective feedback from environment reward and execution rather than teleoperation demonstrations. It generalizes to perturbed pick-and-place settings like LIBERO-PRO, where OpenVLA scores 0.0% and π0.5 averages 12.83%. Using the same low-level primitives, RHO reaches a 45.0% success rate, 2.5x higher than the strongest multi-turn agentic system, and 3.5x higher than π0.5. On Robosuite, RHO sets a new state-of-the-art of 70.0%, exceeding the prior multi-turn record of 68.29% using single-turn execution with no corrective LLM code edits at deployment. When an LLM is used in the control loop, as on RAI's O3DE benchmark, RHO optimizes the deployed agent's multi-file harness of prompts, tools, and control code, improving held-out success from 23.5% to 44.3% with 20% less wall-clock time and 27% fewer tool calls.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 85 of its 95 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

85 of 95 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation23 Mar
  2. 2026Act-Observe-Rewrite: Multimodal Coding Agents as In-Context Policy Learners for Robot Manipulation3 Mar
  3. 2026Demonstration-Free Robotic Control via LLM Agents28 Jan
  4. 2025MAESTRO: Multi-Agent Environment Shaping through Task and Reward Optimization24 Nov
  5. 2025RAI: Flexible Agent Framework for Embodied AI12 May

Other references · 80

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  4. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  7. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  10. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  11. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  12. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
38 more
  1. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  2. 2023Mastering Diverse Domains through World Modelscited by 1,193
  3. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  4. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  5. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  6. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  7. 2025SAM 3: Segment Anything with Conceptscited by 835
  8. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  9. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  10. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  11. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  12. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  13. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  14. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  15. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  16. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
  17. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  18. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  19. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
  20. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  21. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 296
  22. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  23. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  24. 2020Integrated Task and Motion Planningcited by 281
  25. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Modelscited by 278
  26. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  27. 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 248
  28. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 219
  29. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  30. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 189
  31. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
  32. 2024RoboDreamer: Learning Compositional World Models for Robot Imaginationcited by 187
  33. 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorizationcited by 166
  34. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 155
  35. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  36. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 126
  37. 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 120
  38. 2023Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Modelcited by 115

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 18.

18 papers.

Sort:

By area:

  1. 2026EmbodiedSWE: Coding Agents for Long Horizon Dexterous Roboticscited by 2
  2. 2026You Don't Need To Stay in The Loop: An Agentic Robotics Loop for Robot-Policy Improvementcited by 1
  3. 2026Coding Agents for Generalized Task and Motion Planning Problemscited by 1
  4. 2026RLE-Bench: A Qualifying Exam for Coding Agents as Robot Learning Engineerscited by 1
  5. 2026Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agentscited by 1
  6. 2026Skills in Weights, Memory in Code: Hybrid Learning for Memory-Dependent Robot Manipulationcited by 0
  7. 2026Loreley: Repository-Scale Program Evolution with Quality-Diversity Searchcited by 0
  8. 2026Learning and Transferring Closed-Loop Robot Softwarecited by 0
  9. 2026From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Interventioncited by 0
  10. 2026What Stops Recursive Self-Improvement in Robotics? Lessons from 123 Rounds of Agentic Skill Discoverycited by 0
  11. 2026Beyond Scripted Search: Sample-Efficient Reward Discovery via Agentic Black-box Optimizationcited by 0
  12. 2026MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulationcited by 0
6 more
  1. 2026Make Code as Policy Great Again: Frontier Agents Write, Call, and Evolve Robot Toolscited by 0
  2. 2026Scale and Selection: What Makes Automatic Harness Evolution Work for Visual-Interface Robot Agentscited by 0
  3. 2026DynaHarness: A Dynamic Physical Harness for Self-Evolving Robot Agentscited by 0
  4. 2026InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulationcited by 0
  5. 2026RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomescited by 0
  6. 2026Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Nautilus: From One Prompt to Plug-and-Play Robot Learning
  2. 2026Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligence
  3. 2026LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents
  4. 2026RACaP: Agentic Reasoning, Acting, and Coding as Policies for Evolvable Robot Learning
  5. 2026ENPIRE: Agentic Robot Policy Self-Improvement in the Real World
  6. 2025RoboInspector: Unveiling the Unreliability of Policy Code for LLM-enabled Robotic Manipulation
  7. 2026CodeMidas: Scaling Agentic Coding RL Environments from Code Itself
  8. 2026OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies