# RoboProcessBench: Benchmarking Process-Aware Understanding in Vision-Language Robotic Manipulation

arXiv: https://arxiv.org/abs/2606.13040 · HTML: https://mlsanity.com/paper/2606.13040/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- First posted on arXiv 11 Jun 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Built on by (most cited first, 1 of 1)

- [H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models](https://mlsanity.com/paper/2608.13049/) (2026)

## Abstract (arXiv)

Vision-language models (VLMs) used as robotic critics, reward generators, and failure detectors must understand how manipulation unfolds, not only whether it succeeds. Yet fine-grained process supervision is not readily available in a unified form, and existing evaluations do not systematically diagnose these abilities. We introduce Robo2Process, a trace-grounded framework for scalable construction of process-aware supervision from heterogeneous robotic execution data. It aligns visual observations with source-native signals and annotations, extracts local process units (LPUs), and generates structured QA supervision through task-specific sampling, deterministic label construction, and ambiguity filtering. We instantiate this framework as RoboProcessBench, a diagnostic benchmark with 12 question families spanning static monitoring and dynamic reasoning, and ProcessData, a corpus of 57,892 QA pairs across four data sources and 260 manipulation tasks, split into ProcessData-SFT and ProcessData-Eval. Evaluation of a broad suite of VLMs reveals uneven process understanding. Supervision from ProcessData-SFT improves textitQwen2.5-VL-7B and textitInternVL-3-8B on local state, motion, progress, and primitive-aware cues. External evaluation on RoboRewardBench shows Robo2Process enables out-of-distribution process-aware reasoning abilities. Overall, Robo2Process provides a source-aware pipeline, diagnostic benchmark, and learnable substrate for process-aware understanding in robotic manipulation.