# AR4D: Autoregressive 4D Generation from Monocular Videos

arXiv: https://arxiv.org/abs/2501.01722 · HTML: https://mlsanity.com/paper/2501.01722/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jan 2025.

## Built on by (most cited first, 16 of 16)

- [SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generation](https://mlsanity.com/paper/2503.16396/) (2025)
- [Puppeteer: Rig and Animate Your 3D Models](https://mlsanity.com/paper/2508.10898/) (2025)
- [Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation](https://mlsanity.com/paper/2509.19296/) (2025)
- [AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Models](https://mlsanity.com/paper/2506.19851/) (2025)
- [Vivid4D: Improving 4D Reconstruction from Monocular Video by Video Inpainting](https://mlsanity.com/paper/2504.11092/) (2025)
- [Can Video Diffusion Model Reconstruct 4D Geometry?](https://mlsanity.com/paper/2503.21082/) (2025)
- [Stable Part Diffusion 4D: Multi-View RGB and Kinematic Parts Video Generation](https://mlsanity.com/paper/2509.10687/) (2025)
- [Advances in 4D Representation: Geometry, Motion, and Interaction](https://mlsanity.com/paper/2510.19255/) (2025)
- [BiMotion: B-spline Motion for Text-guided Dynamic 3D Character Generation](https://mlsanity.com/paper/2602.18873/) (2026)
- [TwoSquared: 4D Generation from 2D Image Pairs](https://mlsanity.com/paper/2504.12825/) (2025)
- [SWiT-4D: Sliding-Window Transformer for Lossless and Parameter-Free Temporal 4D Generation](https://mlsanity.com/paper/2512.10860/) (2025)
- [CP4D: Compositional Physics-aware 4D Scene Generation](https://mlsanity.com/paper/2606.09187/) (2026)
- [Advances in 4D Generation: A Survey](https://mlsanity.com/paper/2503.14501/) (2025)
- [Feed-forward Motion In-betweening for Any 4D](https://mlsanity.com/paper/2606.22131/) (2026)
- [Follow Your Track: Precise Skeleton Animation Controlled by 3D Trajectories](https://mlsanity.com/paper/2606.25344/) (2026)
- [Streaming4D: Accelerate 4D World Models via Block-wise Video Generation and Incremental Reconstruction](https://mlsanity.com/paper/2609.00610/) (2026)

## Abstract (arXiv)

Recent advancements in generative models have ignited substantial interest in dynamic 3D content creation (ie, 4D generation). Existing approaches primarily rely on Score Distillation Sampling (SDS) to infer novel-view videos, typically leading to issues such as limited diversity, spatial-temporal inconsistency and poor prompt alignment, due to the inherent randomness of SDS. To tackle these problems, we propose AR4D, a novel paradigm for SDS-free 4D generation. Specifically, our paradigm consists of three stages. To begin with, for a monocular video that is either generated or captured, we first utilize pre-trained expert models to create a 3D representation of the first frame, which is further fine-tuned to serve as the canonical space. Subsequently, motivated by the fact that videos happen naturally in an autoregressive manner, we propose to generate each frame's 3D representation based on its previous frame's representation, as this autoregressive generation manner can facilitate more accurate geometry and motion estimation. Meanwhile, to prevent overfitting during this process, we introduce a progressive view sampling strategy, utilizing priors from pre-trained large-scale 3D reconstruction models. To avoid appearance drift introduced by autoregressive generation, we further incorporate a refinement stage based on a global deformation field and the geometry of each frame's 3D representation. Extensive experiments have demonstrated that AR4D can achieve state-of-the-art 4D generation without SDS, delivering greater diversity, improved spatial-temporal consistency, and better alignment with input prompts.