mlsanityData to 14 Oct 2026

RoboChallenge: Large-scale Real-robot Evaluation of Embodied Policies

arXiv 2510.17950 ↗First posted 20 Oct 2025cs.ROcited by 30

Not on the map. Why?

Abstract (arXiv)

Testing on real machines is indispensable for robotic control algorithms. In the context of learning-based algorithms, especially VLA models, demand for large-scale evaluation, i.e. testing a large number of models on a large number of tasks, is becoming increasingly urgent. However, doing this right is highly non-trivial, especially when scalability and reproducibility is taken into account. In this report, we describe our methodology for constructing RoboChallenge, an online evaluation system to test robotic control algorithms, and our survey of recent state-of-the-art VLA models using our initial benchmark Table30.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Oct 2025.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 30.

30 papers.

Sort:

By area:

  1. 2026A Pragmatic VLA Foundation Modelcited by 103
  2. 2026World Action Models: The Next Frontier in Embodied AIcited by 50
  3. 2026RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policiescited by 40
  4. 2026StarVLA-α: Reducing Complexity in Vision-Language-Action Systemscited by 31
  5. 2026Towards Generalizable Robotic Manipulation in Dynamic Environmentscited by 20
  6. 2026Wall-OSS-0.5 Technical Reportcited by 12
  7. 2026World Action Models: A Surveycited by 12
  8. 2026GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluationcited by 11
  9. 2026ProcVLM: Learning Procedure-Grounded Progress Rewards for Robotic Manipulationcited by 9
  10. 2026Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Enginescited by 8
  11. 2026PACE: Phase-Aware Chunk Execution for Robot Policies with Action Chunkingcited by 8
  12. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 8
18 more
  1. 2026IG-RFT: An Interaction-Guided RL Framework for VLA Models in Long-Horizon Robotic Manipulationcited by 5
  2. 2026ManipArena: Comprehensive Real-world Evaluation of Reasoning-Oriented Generalist Robot Manipulationcited by 5
  3. 2026LABSHIELD: A Multimodal Benchmark for Safety-Critical Reasoning and Planning in Scientific Laboratoriescited by 3
  4. 2026PRM-as-a-Judge 1.5: A Toolkit for Robot Process Assessmentcited by 3
  5. 2026VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Modelscited by 2
  6. 2026PhAIL: A Real-Robot VLA Benchmark and Distributional Methodologycited by 2
  7. 2026You Don't Need To Stay in The Loop: An Agentic Robotics Loop for Robot-Policy Improvementcited by 2
  8. 2026X-DiffVLA: X-Embodied Diffusion Action Heads for Vision-Language-Action Modelscited by 1
  9. 2026Intelligent Automation for Embodied Benchmark Construction: Pipelines, Embodiments, Simulators, and Trendscited by 1
  10. 2026ATOM-Bench: A Real-World Benchmark for Atomic Skills and Compositional Generalization in Manipulation Policiescited by 1
  11. 2026RoboFollow: Unveiling the Instruction Following Mirage in Embodied Agentscited by 1
  12. 2026Grounding Sim-to-Real Generalization in Robotic Manipulation: An Empirical Study with Vision-Language-Action Modelscited by 0
  13. 2026PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulationcited by 0
  14. 2026Towards Trustworthy Embodied Intelligence: A Systems Framework and Graded Trustworthiness Levelscited by 0
  15. 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0
  16. 2026One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulationcited by 0
  17. 2026RoboFolDeX: A Physical-World Benchmark for Long-Horizon Robotic Manipulation of Deformable Objectscited by 0
  18. 2026EWAM: Emergent Depth-Wise Specialization in a Unified Embodied Model -- From Semantic Understanding through Visual Foresight to Actioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies
  2. 2025Reliable and Scalable Robot Policy Evaluation with Imperfect Simulators
  3. 2026IndustrialVLA-Bench: A Traceable Multi-Axis Evaluation of Open Robot Policy Models
  4. 2026RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation
  5. 2026ArmnetBench v0.1: Parallel Real-World Evaluation of Manipulation Policies on a Low-Cost Arm Farm
  6. 2025WorldGym: World Model as An Environment for Policy Evaluation
  7. 2026RoboSPA: Can VLA Models Go Beyond Simple Scenes and Short-Horizon Tasks?
  8. 2026R2S-Eval: Robot Evaluation with Real-to-Sim Calibration via Vision-Language Models