Sampling headroom is not selection gain: a compute-value audit of test-time scaling for video world models
On the map: World models → Video-generation world models and simulators
also in 1 more
Abstract (arXiv)
Test-time scaling (TTS) can improve generation only when additional compute produces better candidates and the system can reliably identify them. This distinction is especially important for video world models, where a wider sample pool may contain stronger rollouts without improving the output that is ultimately selected. We introduce the Compute-Value Audit (CVA), a sequential framework that asks whether extra sampling creates opportunity, observable signals provide a reliable state, that state supports a beneficial action, and the resulting gain exceeds the full entry fee of generation and verification. On 192 Physics-IQ scenes, expanding the pool from 4 to 16 candidates increases oracle quality by +9.23 IQ (95% CI [+7.44, +11.14]), but Flow, Cycle, and VideoReward fail to recover this headroom reliably. Across three generators, none of twelve adaptive-depth policies outperforms uniform compute; they recover only 42-69% of the measured entry fee. A matched-60-NFE Predict-and-Perturb intervention on VideoPhy2 is likewise negative across three fresh-seed replicas. These negative results are not universal: anchor-explorer passes all four stages in a sparse PRM800K setting, MMLU-Pro exposes the gap between predictive state and useful action, and a privileged paired future establishes a positive video upper bound. Together, these results show that sampling headroom has deployment value only when it can be converted into a reliable decision whose benefit survives the complete compute charge. Code is available at https://github.com/YuhuaJiang2002/sampling-headroom-is-not-selection-gain.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 50 of its 56 references that are in the map. First posted on arXiv 6 Sep 2026.
Builds on
50 of 56 references.
Closest ancestors · 3
Same direction, newest first
- 2026Off-Manifold Refinement: Guiding Video Generators with a Frozen World Model30 Aug
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2022Flow Matching for Generative Modeling6 Oct
Other references · 47
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Let's Verify Step by Stepcited by 3,598
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
35 more
- 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
- 2023Reasoning with Language Model is Planning with World Modelcited by 846
- 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
- 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 707
- 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
- 2022Continual Test-Time Domain Adaptationcited by 520
- 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
- 2025Improving Video Generation with Human Feedbackcited by 267
- 2018Residual Reinforcement Learning for Robot Controlcited by 242
- 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
- 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 225
- 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 215
- 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 201
- 2024VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generationcited by 190
- 2023Robust Test-Time Adaptation in Dynamic Scenarioscited by 185
- 2025Do generative video models understand physical principles?cited by 151
- 2017Neural Episodic Controlcited by 148
- 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generationcited by 148
- 2017Safe Policy Improvement with Baseline Bootstrappingcited by 141
- 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 141
- 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 135
- 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
- 2025Video-T1: Test-Time Scaling for Video Generationcited by 39
- 2022Semi-Parametric Neural Image Synthesiscited by 36
- 2026Inference-time Physics Alignment of Video Generative Models with Latent World Modelscited by 32
- 2026Self-Refining Video Samplingcited by 14
- 2026Adaptive Test-Time Compute Allocation for Reasoning LLMs via Constrained Policy Optimizationcited by 8
- 2025Planning with Sketch-Guided Verification for Physics-Aware Video Generationcited by 6
- 2026Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generationcited by 6
- 2026Retrieve-then-Steer: Online Success Memory for Test-Time Adaptation of Generative VLAscited by 5
- 2026AdaMEM: Test-Time Adaptive Memory for Language Agentscited by 4
- 2026Stream-T1: Test-Time Scaling for Streaming Video Generationcited by 3
- 2026ComMem: Complementary Memory Systems for Test-Time Adaptation of Vision-Language Modelscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Test-Time Scaling for Video Diffusion Models via Diagnosis-Guided Candidate Recycling
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity
- 2025ScalingNoise: Scaling Inference-Time Search for Generating Infinite Videos
- 2025WorldModelBench: Judging Video Generation Models As World Models
- 2026SCOPE: Score-Isolated Agentic Optimization for Video World Models
- 2026MemoBench: Benchmarking World Modeling in Dynamically Changing Environments
- 2026WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors
- 2026Physics-IQ Verified