# AnyView: Synthesizing Any Novel View in Dynamic Scenes

arXiv: https://arxiv.org/abs/2601.16982 · HTML: https://mlsanity.com/paper/2601.16982/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 75 references that are in the map.
- First posted on arXiv 23 Jan 2026.

## Builds on (most cited first, 30 of 51)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision](https://mlsanity.com/paper/2312.16256/) (2023)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [CameraCtrl: Enabling Camera Control for Text-to-Video Generation](https://mlsanity.com/paper/2404.02101/) (2024)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion](https://mlsanity.com/paper/2403.12008/) (2024)
- [GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control](https://mlsanity.com/paper/2503.03751/) (2025)
- [MapAnything: Universal Feed-Forward Metric 3D Reconstruction](https://mlsanity.com/paper/2509.13414/) (2025)
- [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022)
- [ReCamMaster: Camera-Controlled Generative Rendering from A Single Video](https://mlsanity.com/paper/2503.11647/) (2025)
- [PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking](https://mlsanity.com/paper/2307.15055/) (2023)
- [Shape of Motion: 4D Reconstruction from a Single Video](https://mlsanity.com/paper/2407.13764/) (2024)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos](https://mlsanity.com/paper/2409.02095/) (2024)
- [World Simulation with Video Foundation Models for Physical AI](https://mlsanity.com/paper/2511.00062/) (2025)
- [MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffolds](https://mlsanity.com/paper/2405.17421/) (2024)
- [ViPE: Video Pose Engine for 3D Geometric Perception](https://mlsanity.com/paper/2508.10934/) (2025)
- [Ctrl-World: A Controllable Generative World Model for Robot Manipulation](https://mlsanity.com/paper/2510.10125/) (2025)

## Built on by (most cited first, 12 of 12)

- [AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories](https://mlsanity.com/paper/2602.14941/) (2026)
- [Full-4D: Generating Full-Scope 4D Scenes from a Single-View Video](https://mlsanity.com/paper/2605.25500/) (2026)
- [FrameCrafter: Novel View Synthesis as Video Completion](https://mlsanity.com/paper/2604.08500/) (2026)
- [RoboDream: Compositional World Models for Scalable Robot Data Synthesis](https://mlsanity.com/paper/2606.02577/) (2026)
- [MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation](https://mlsanity.com/paper/2606.26087/) (2026)
- [OpenLongTail: Generative Scaling of Long-Tail Driving Data](https://mlsanity.com/paper/2607.09655/) (2026)
- [VideoTok4D: A 4D-Aware Video Tokenizer for Compact World Representation](https://mlsanity.com/paper/2609.12874/) (2026)
- [Flex4DHuman: Flexible Multi-view Video Diffusion for 4D Human Reconstruction](https://mlsanity.com/paper/2606.13655/) (2026)
- [MetaView: Monocular Novel View Synthesis with Scale-Aware Implicit Geometry Priors](https://mlsanity.com/paper/2607.12000/) (2026)
- [Can 4D Foundation Models Remember?](https://mlsanity.com/paper/2609.20819/) (2026)
- [VGGT-Diff: Visual Geometry Meets Diffusion for Sparse-View Novel View Synthesis](https://mlsanity.com/paper/2609.33253/) (2026)
- [Multi-Agent Egocentric World Model with Fine-Grained Embodied Interaction](https://mlsanity.com/paper/2610.12299/) (2026)

## Abstract (arXiv)

Modern generative video models excel at producing convincing, high-quality outputs, but struggle to maintain multi-view and spatiotemporal consistency in highly dynamic real-world environments. In this work, we introduce AnyView, a diffusion-based video generation framework for textitdynamic view synthesis with minimal inductive biases or geometric assumptions. We leverage multiple data sources with various levels of supervision, including monocular (2D), multi-view static (3D) and multi-view dynamic (4D) datasets, to train a generalist spatiotemporal implicit representation capable of producing zero-shot novel videos from arbitrary camera locations and trajectories. We evaluate AnyView on standard benchmarks, showing competitive results with the current state of the art, and propose AnyViewBench, a challenging new benchmark tailored towards textitextreme dynamic view synthesis in diverse real-world scenarios. In this more dramatic setting, we find that most baselines drastically degrade in performance, as they require significant overlap between viewpoints, while AnyView maintains the ability to produce realistic, plausible, and spatiotemporally consistent videos when prompted from textitany viewpoint. Results, data, code, and models can be viewed at: https://tri-ml.github.io/AnyView/