# PiL-World: A Chunk-Wise World Model for VLA Policy-in-the-Loop Evaluation

arXiv: https://arxiv.org/abs/2606.05773 · HTML: https://mlsanity.com/paper/2606.05773/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)
- Vision-language-action models / [World model-based VLA](https://mlsanity.com/vla/world-model-based-vla/)

## Builds on (most cited first, 18 of 18)

- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Evaluating Real-World Robot Manipulation Policies in Simulation](https://mlsanity.com/paper/2405.05941/) (2024)
- [WorldVLA: Towards Autoregressive Action World Model](https://mlsanity.com/paper/2506.21539/) (2025)
- [Ctrl-World: A Controllable Generative World Model for Robot Manipulation](https://mlsanity.com/paper/2510.10125/) (2025)
- [IRASim: A Fine-Grained World Model for Robot Manipulation](https://mlsanity.com/paper/2406.14540/) (2024)
- [WorldEval: World Model as Real-World Robot Policies Evaluator](https://mlsanity.com/paper/2505.19017/) (2025)
- [WorldGym: World Model as An Environment for Policy Evaluation](https://mlsanity.com/paper/2506.00613/) (2025)
- [VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World Model](https://mlsanity.com/paper/2602.12063/) (2026)
- [AVID: Adapting Video Diffusion Models to World Models](https://mlsanity.com/paper/2410.12822/) (2024)
- [Improvisation through Physical Understanding: Using Novel Objects as Tools with Visual Foresight](https://mlsanity.com/paper/1904.05538/) (2019)
- [ABot-PhysWorld: Interactive World Foundation Model for Robotic Manipulation with Physics Alignment](https://mlsanity.com/paper/2603.23376/) (2026)
- [dWorldEval: Scalable Robotic Policy Evaluation via Discrete Diffusion World Model](https://mlsanity.com/paper/2604.22152/) (2026)
- [Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training](https://mlsanity.com/paper/2604.21741/) (2026)
- [Towards High-Consistency Embodied World Model with Multi-View Trajectory Videos](https://mlsanity.com/paper/2511.12882/) (2025)

## Built on by (most cited first, 2 of 2)

- [Do Robotic World Models Really Follow Actions? Diagnosing and Aligning Action-Conditioned Generation for Policy Learning](https://mlsanity.com/paper/2608.24885/) (2026)
- [Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics](https://mlsanity.com/paper/2610.13522/) (2026)

## Abstract (arXiv)

Vision-language-action (VLA) policies operate in a closed loop in real-world robot tasks: a robot observes the scene, executes an action chunk, and conditions its next decision on the resulting observation. However, most existing world models for robot action evaluation are limited to open-loop prediction along pre-collected action trajectories. This prevents them from supporting closed-loop VLA evaluation, where each action chunk must be conditioned on the observation generated by the previous execution. To address this gap, we propose PiL-World, a chunk-wise world model designed for policy-in-the-loop VLA evaluation. Given the current observation and the action trajectory rolled out by a VLA policy, PiL-World generates multi-view future observations that are consistent with the VLA rollout and match the image inputs required by the policy. By alternating between VLA inference and world-model prediction, PiL-World enables closed-loop evaluation without real robot execution at every step. To improve rollout fidelity, PiL-World conditions video generation on action-derived visual control from head-view robot motion and latent histories that encode task execution context, while jointly predicting complementary multi-view observations. Beyond successful teleoperated demonstrations, it also learns from failed execution trajectories, helping the imagined rollouts better match the distribution of real policy executions. We evaluate PiL-World on three real dual-arm manipulation tasks. PiL-World generates imagined rollouts that are highly consistent with real robot executions. More importantly, compared with the baseline, it reduces the error between VLA success rates measured in real-world rollouts and those estimated through closed-loop world-model evaluation from 63.2% to 12.0%.