Can a Frontier VLM Serve as a Robot Policy for Memory-Dependent Manipulation?
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
We use a frozen frontier vision-language model, GPT-6-Astra, directly as the robot policy on RoboMME, a benchmark of 16 memory-dependent manipulation tasks spanning temporal, spatial, object, and procedural memory. The model keeps the whole episode in context and outputs every end-effector waypoint and gripper command, with no RoboMME-specific training. On the 800 test episodes it succeeds in 92.1%, above the previously reported human score of 90.5%, which was obtained with an oracle skill interface, and far ahead of the best leaderboard entry (60.8%) and of a hierarchical system that pairs the same model with a RoboMME-trained policy (79.1%). It matches or exceeds the human score on 13 of the 16 tasks. Its 63 failures concentrate in two tasks and are mostly premature completions and mistimed or unintended button presses. These results blur the usual division between long-horizon planning, memory and end-effector control, and raise the question of how much of this stack a single general-purpose model can take over in embodied systems. Project website: https://embodied-astra.github.io. Code, prompts and all episode logs: https://github.com/jackbjed/embodied-astra-robomme.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. First posted on arXiv 12 Oct 2026.
Builds on
No reference list yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies
- 2026ECoMEM: Explicit Concept Memory for Memory-Dependent Robot Control ↗
- 2026AGM: Achievement-Grounded Memory for Closed-Loop Agents with Frozen VLA Policies
- 2026T2Mem: Learning Test-Time Memory for Robotics ↗
- 2024What Matters in Building Vision-Language-Action Models for Generalist Robots
- 2026OnEvoMemory: Evolving Memory through Online Robot Rollouts for Pretrained Robot Policies ↗
- 2026MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation
- 2026RoboMME-Interference: Benchmarking Robot Memory Under Interference