ARCON: Advancing Auto-Regressive Continuation for Driving Videos
On the map: End-to-end autonomous driving → Generative simulation for driving
Abstract (arXiv)
Front-view driving video continuation is a critical component for constructing sophisticated world models. However, maintaining long-term coherence faces the fundamental challenge of mitigating ''generative degeneration'' during the auto-regressive process. This phenomenon arises when the limited attention capacity of world models becomes saturated with high-entropy pixel details over extended sequences, leading to a collapse of global scene structure and motion intensity. To address this, we introduce STRIDE (STructural RegIsters for Decoupled Extrapolation), a large vision model architecture with an enforced semantic foresight mechanism. STRIDE decouples video continuation into two sub-tasks via interleaved generation of semantic and RGB tokens: forecasting low-entropy semantic maps, and subsequently generating high-frequency visual appearance conditioned on this structure. The semantic tokens explicitly function as ''structural registers'' -- dedicated memory slots that robustly retain the scene's long-term context and world dynamics. Furthermore, we employ an architecturally decoupled flow matching-based super-resolution module to enhance the final visual fidelity. Extensive experiments in autonomous driving scenarios demonstrate that STRIDE effectively mitigates generative degeneration, achieving minute-level, temporally consistent driving video continuation with strong generalization ability.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving. Builds on 69 of its 123 references that are in the map. First posted on arXiv 4 Dec 2024.
Builds on
69 of 123 references.
Closest ancestors · 5
Same direction, newest first
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
- 2024GenAD: Generalized Predictive Model for Autonomous Driving14 Mar
- 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
- 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
Other references · 64
- 2017Attention Is All You Needcited by 59,932
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2017Neural Discrete Representation Learningcited by 4,650
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2021Zero-Shot Text-to-Image Generationcited by 4,276
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
- 2019The Curious Case of Neural Text Degenerationcited by 2,881
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
38 more
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
- 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
- 2022MaskGIT: Masked Generative Image Transformercited by 1,082
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
- 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2023Vision Transformers Need Registerscited by 844
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
- 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
- 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
- 2019Neural Text Generation with Unlikelihood Trainingcited by 489
- 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
- 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2023Generative Multimodal Models are In-Context Learnerscited by 446
- 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
- 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
- 2022MAGVIT: Masked Generative Video Transformercited by 341
- 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
- 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 317
- 2023Photorealistic Video Generation with Diffusion Modelscited by 304
- 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
- 2024Diffusion Models Are Real-Time Game Enginescited by 282
- 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Textcited by 242
- 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
- 2022UniFormer: Unifying Convolution and Self-attention for Visual Recognitioncited by 206
- 2023A Survey on Video Diffusion Modelscited by 203
- 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 165
- 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 162
- 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 153
- 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 128
- 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
- 2022MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generationcited by 125
50 most cited of 64 All 64 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
- 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT
- 2026Representations Before Pixels: Semantics-Guided Hierarchical Video Prediction
- 2025STAGE: A Stream-Centric Generative World Model for Long-Horizon Driving-Scene Simulation
- 2026Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving
- 2026FAR-Drive: Frame-AutoRegressive Video Generation in Closed-Loop Autonomous Driving
- 2026VistaGEN: Consistent Driving Video Generation with Fine-Grained Control Using Multiview Visual-Language Reasoning
- 2025Long-Context Autoregressive Video Modeling with Next-Frame Prediction