mlsanityData to 7 Oct 2026

Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligence

arXiv 2609.35432 ↗First posted 28 Sep 2026cs.ROcited by 0

On the map: Vision-language-action models → Failure recovery and robustness also fits: World model-based VLA

also in 1 more
Abstract (arXiv)

Vision-language-action (VLA) and world-action (WAM) models map observations and instructions directly to robot actions. This directness ties a policy to training: minor layout or viewpoint changes cause failure, and instructions generalize poorly. The root cause lies in representation: task requirements, conditions, progress, and failure recovery are implicitly encoded in action sequences, making them difficult to inspect or revise. Digital coding agents offer a precedent: LLMs call tools, verify results, and revise from feedback as executable code. The same working pattern of explicit state, manageable execution, and revisable procedures underlies generalization and long-horizon execution in the physical world, letting physical experience return as reusable programs, memory, or evidence. We propose Physical Coding, representing task state and execution as code. Code as World records objects, relations, constraints, and progress; Code as Policy organizes planning, verification, recovery, and execution. We build HexaAnything, which calls perception, planning, and control tools, including VLA/WAM policies, and makes in-the-loop decisions from external feedback. Verified traces become data and memory, enabling evolution from tools and Harness to model weights, architectures, and ultimately hardware and task design. On RoboCasa365, HexaAnything improves Composite-Unseen and overall success over XR-1 VLA, and its Harness-trained HexaModel beats the base on every split, indicating code traces internalize physical execution. On PhyBench and a dual-arm AgileX robot, the agent autonomously completes physics experiments and most tabletop tasks, often faster than published results. We observe data, model, and tool self-evolution; future work targets weight internalization, autonomous redesign of architectures, languages, representations, and tasks, and deployment in manufacturing and science.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Failure recovery and robustness, one of 13 directions in Vision-language-action models. Builds on 99 of its 109 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

99 of 109 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation23 Mar
  2. 2026LangForce: Bayesian Decomposition of Vision Language Action Models via Latent Action Queries21 Jan
  3. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models15 Oct
  4. 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization4 Oct

Other references · 95

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2025Qwen3-VL Technical Reportcited by 2,534
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  9. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  10. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  11. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  12. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
38 more
  1. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  2. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  3. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  4. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  5. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  6. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  7. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  8. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  9. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
  10. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  11. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  12. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  13. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  14. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  15. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  16. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  17. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  18. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  19. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  20. 2023Learning Interactive Real-World Simulatorscited by 440
  21. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
  22. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
  23. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  24. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  25. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  26. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  27. 2026Causal World Modeling for Robot Controlcited by 289
  28. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 260
  29. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  30. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  31. 2025KernelBench: Can LLMs Write Efficient GPU Kernels?cited by 206
  32. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 204
  33. 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 163
  34. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  35. 2025Do generative video models understand physical principles?cited by 151
  36. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 146
  37. 2024Vision language models are blind: Failing to translate detailed visual features into wordscited by 144
  38. 2023Vision-Language Models as Success Detectorscited by 138

50 most cited of 95 All 95 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026RLE-Bench: A Qualifying Exam for Coding Agents as Robot Learning Engineers
  2. 2026RACaP: Agentic Reasoning, Acting, and Coding as Policies for Evolvable Robot Learning
  3. 2026Encore: Few-Shot Agentic Discovery of Manipulation Strategies
  4. 2026Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
  5. 2026Act-Observe-Rewrite: Multimodal Coding Agents as In-Context Policy Learners for Robot Manipulation
  6. 2026EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents
  7. 2026RHO: Your Coding Agent is Secretly a Roboticist
  8. 2026Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures