# Entropy-Guided k-Guard Sampling for Long-Horizon Autoregressive Video Generation

arXiv: https://arxiv.org/abs/2601.19488 · HTML: https://mlsanity.com/paper/2601.19488/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 41 references that are in the map.
- First posted on arXiv 27 Jan 2026.

## Builds on (most cited first, 30 of 36)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [GAIA-1: A Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2309.17080/) (2023)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [Phenaki: Variable Length Video Generation From Open Domain Textual Description](https://mlsanity.com/paper/2210.02399/) (2022)
- [Scaling Autoregressive Video Models](https://mlsanity.com/paper/1906.02634/) (2019)
- [Scalable Best-of-N Selection for Large Language Models via Self-Certainty](https://mlsanity.com/paper/2502.18581/) (2025)
- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [iVideoGPT: Interactive VideoGPTs are Scalable World Models](https://mlsanity.com/paper/2405.15223/) (2024)
- [Generalization in Generation: A closer look at Exposure Bias](https://mlsanity.com/paper/1910.00292/) (2019)
- [Long-Context Autoregressive Video Modeling with Next-Frame Prediction](https://mlsanity.com/paper/2503.19325/) (2025)
- [Epona: Autoregressive Diffusion World Model for Autonomous Driving](https://mlsanity.com/paper/2506.24113/) (2025)
- [WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens](https://mlsanity.com/paper/2401.09985/) (2024)
- [Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generation](https://mlsanity.com/paper/2502.20388/) (2025)
- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT](https://mlsanity.com/paper/2412.19505/) (2024)
- [Confidence Regulation Neurons in Language Models](https://mlsanity.com/paper/2406.16254/) (2024)
- [Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics](https://mlsanity.com/paper/2501.10100/) (2025)
- [A Survey: Learning Embodied Intelligence from Physical Simulators and World Models](https://mlsanity.com/paper/2507.00917/) (2025)
- [EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling](https://mlsanity.com/paper/2403.14541/) (2024)
- [Pre-Trained Video Generative Models as World Simulators](https://mlsanity.com/paper/2502.07825/) (2025)
- [Conformal Nucleus Sampling](https://mlsanity.com/paper/2305.02633/) (2023)
- [Taming Teacher Forcing for Masked Autoregressive Video Generation](https://mlsanity.com/paper/2501.12389/) (2025)
- [Next Block Prediction: Video Generation via Semi-Autoregressive Modeling](https://mlsanity.com/paper/2502.07737/) (2025)
- [Lumos-1: On Autoregressive Video Generation with Discrete Diffusion from a Unified Model Perspective](https://mlsanity.com/paper/2507.08801/) (2025)
- [Error Analyses of Auto-Regressive Video Diffusion Models: A Unified Framework](https://mlsanity.com/paper/2503.10704/) (2025)

## Abstract (arXiv)

Autoregressive (AR) architectures have achieved significant successes in LLMs, inspiring explorations for video generation. In LLMs, top-p/top-k sampling strategies work exceptionally well: language tokens have high semantic density and low redundancy, so a fixed size of token candidates already strikes a balance between semantic accuracy and generation diversity. In contrast, video tokens have low semantic density and high spatio-temporal redundancy. This mismatch makes static top-k/top-p strategies ineffective for video decoders: they either introduce unnecessary randomness for low-uncertainty regions (static backgrounds) or get stuck in early errors for high-uncertainty regions (foreground objects). Prediction errors will accumulate as more frames are generated and eventually severely degrade long-horizon quality. To address this, we propose Entropy-Guided k-Guard (ENkG) sampling, a simple yet effective strategy that adapts sampling to token-wise dispersion, quantified by the entropy of each token's predicted distribution. ENkG uses adaptive token candidate sizes: for low-entropy regions, it employs fewer candidates to suppress redundant noise and preserve structural integrity; for high-entropy regions, it uses more candidates to mitigate error compounding. ENkG is model-agnostic, training-free, and adds negligible overhead. Experiments demonstrate consistent improvements in perceptual quality and structural stability compared to static top-k/top-p strategies.