mlsanityData to 14 Oct 2026

HarnessPAI: An Evolving Harness for Physical AI

arXiv 2609.29166 ↗First posted 24 Sep 2026cs.RO cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Physical AI aims to build embodied agents that perceive the world, understand and reason about it, and decide how to act. Yet the field has focused primarily on the last component: the action model that maps observations to low-level controls. The prevailing training recipe can erode the perceptual and reasoning capabilities needed for robust behavior, leaving even strong action models vulnerable to scene perturbations and long-horizon tasks. We introduce HarnessPAI, a model- and embodiment-agnostic Harness framework for Physical AI that treats code as the executable and evolvable interface that organizes the underlying action primitive. The framework separates two timescales: within a rollout, it executes open-loop at the program level, with a fixed program guiding and checking execution; across rollouts, it evolves closed-loop, using execution feedback to revise the program and distill failures into reusable skills. Across desktop robot arms, household robots, a robot vacuum, and a legged walking agent, HarnessPAI improves on both pure action models and code-as-policy baselines without retraining the underlying model: a 61.6-point gain over π0.5 on LIBERO-PRO and a 27.2-point gain over WorldDreamer on RoboCasa atomic tasks. Once a program is selected, rollout execution requires no online high-level LLM deliberation. Beyond execution, the converged program is also a cheap and reliable expert-data collector, and fine-tuning π0.5 on collected expert data lifts success rate on LIBERO-PRO by 38.8 points. Our results suggest that the frontier of Physical AI depends not only on stronger action models, but also on executable harnesses that integrate perception, task understanding and reasoning, and action execution into a unified, verifiable, and feedback-driven system. Website: https://darwin-agent.github.io/HarnessPAI

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 58 of its 63 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

58 of 63 references.

References · 58

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  5. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  6. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  7. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  8. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  9. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  10. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  11. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  12. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
38 more
  1. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  2. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
  3. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  4. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  5. 2026World Action Models are Zero-shot Policiescited by 465
  6. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
  7. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
  8. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 367
  9. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
  10. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 349
  11. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 304
  12. 2025π*0.6: a VLA That Learns From Experiencecited by 303
  13. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 296
  14. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 240
  15. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
  16. 2025MolmoAct: Action Reasoning Models that can Reason in Spacecited by 197
  17. 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorizationcited by 179
  18. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 176
  19. 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planningcited by 156
  20. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 144
  21. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 138
  22. 2025NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Taskscited by 127
  23. 2024GRAPE: Generalizing Robot Policy via Preference Alignmentcited by 109
  24. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokenscited by 89
  25. 2026CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulationcited by 88
  26. 2024DrEureka: Language Model Guided Sim-To-Real Transfercited by 83
  27. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 59
  28. 2025World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Trainingcited by 59
  29. 2026ASPIRE: Agentic /Skills Discovery for Roboticscited by 53
  30. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimizationcited by 49
  31. 2026HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundrycited by 47
  32. 2026Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agentscited by 47
  33. 2026AtomVLA: Scalable Post-Training for Robotic Manipulation via Predictive Latent World Modelscited by 23
  34. 2026Show-Harness: Just a VLM Agent Can Play Robotscited by 20
  35. 2021A Holistic Approach to Reactive Mobile Manipulationcited by 17
  36. 2026Zetta zeta: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligencecited by 9
  37. 2026Memory Intelligence Agentcited by 7
  38. 2026ETA: A New Agentic Paradigm for Embodied Taskscited by 7

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Harness Engineering for Physical AI: Robot Middleware Is the Harness Layer
  2. 2026Towards the Harness of Embodied Agents
  3. 2026Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable
  4. 2026EmbodiedEvo: Self-Evolving Coding Agent For Physical Intelligence
  5. 2026Self-Evolving Embodied Agents via Skill-Harness Evolution
  6. 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
  7. 2026EmbodiedRSI: Active Continual Robot Learning Through Hypothesis-Guided Co-Evolution
  8. 2026Harness-Zero: Harness Distillation via Agent-as-Harness