mlsanityData to 7 Oct 2026

ARCON: Advancing Auto-Regressive Continuation for Driving Videos

arXiv 2412.03758 ↗First posted 4 Dec 2024cs.CVcited by 1

On the map: End-to-end autonomous driving → Generative simulation for driving

Abstract (arXiv)

Front-view driving video continuation is a critical component for constructing sophisticated world models. However, maintaining long-term coherence faces the fundamental challenge of mitigating ''generative degeneration'' during the auto-regressive process. This phenomenon arises when the limited attention capacity of world models becomes saturated with high-entropy pixel details over extended sequences, leading to a collapse of global scene structure and motion intensity. To address this, we introduce STRIDE (STructural RegIsters for Decoupled Extrapolation), a large vision model architecture with an enforced semantic foresight mechanism. STRIDE decouples video continuation into two sub-tasks via interleaved generation of semantic and RGB tokens: forecasting low-entropy semantic maps, and subsequently generating high-frequency visual appearance conditioned on this structure. The semantic tokens explicitly function as ''structural registers'' -- dedicated memory slots that robustly retain the scene's long-term context and world dynamics. Furthermore, we employ an architecturally decoupled flow matching-based super-resolution module to enhance the final visual fidelity. Extensive experiments in autonomous driving scenarios demonstrate that STRIDE effectively mitigates generative degeneration, achieving minute-level, temporally consistent driving video continuation with strong generalization ability.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving. Builds on 69 of its 123 references that are in the map. First posted on arXiv 4 Dec 2024.

Builds on

69 of 123 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
  2. 2024GenAD: Generalized Predictive Model for Autonomous Driving14 Mar
  3. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
  4. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
  5. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep

Other references · 64

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2017Neural Discrete Representation Learningcited by 4,650
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  11. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  12. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
38 more
  1. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  2. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  3. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  4. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  5. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  6. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  7. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  8. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  9. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  10. 2023Vision Transformers Need Registerscited by 844
  11. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  12. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  13. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  14. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  15. 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
  16. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  17. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  18. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  19. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  20. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  21. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  22. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  23. 2022MAGVIT: Masked Generative Video Transformercited by 341
  24. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  25. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 317
  26. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  27. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
  28. 2024Diffusion Models Are Real-Time Game Enginescited by 282
  29. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Textcited by 242
  30. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  31. 2022UniFormer: Unifying Convolution and Self-attention for Visual Recognitioncited by 206
  32. 2023A Survey on Video Diffusion Modelscited by 203
  33. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 165
  34. 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 162
  35. 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 153
  36. 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 128
  37. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  38. 2022MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generationcited by 125

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
  2. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT
  3. 2026Representations Before Pixels: Semantics-Guided Hierarchical Video Prediction
  4. 2025STAGE: A Stream-Centric Generative World Model for Long-Horizon Driving-Scene Simulation
  5. 2026Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving
  6. 2026FAR-Drive: Frame-AutoRegressive Video Generation in Closed-Loop Autonomous Driving
  7. 2026VistaGEN: Consistent Driving Video Generation with Fine-Grained Control Using Multiview Visual-Language Reasoning
  8. 2025Long-Context Autoregressive Video Modeling with Next-Frame Prediction