mlsanityData to 7 Oct 2026

World-Consistent Video-to-Video Synthesis

arXiv 2007.08509 ↗First posted 16 Jul 2020cs.CVcited by 69

Not on the map. Why?

Abstract (arXiv)

Video-to-video synthesis (vid2vid) aims for converting high-level semantic inputs to photorealistic videos. While existing vid2vid methods can achieve short-term temporal consistency, they fail to ensure the long-term one. This is because they lack knowledge of the 3D world being rendered and generate each frame only based on the past few frames. To address the limitation, we introduce a novel vid2vid framework that efficiently and effectively utilizes all past generated frames during rendering. This is achieved by condensing the 3D world rendered so far into a physically-grounded estimate of the current frame, which we call the guidance image. We further propose a novel neural network architecture to take advantage of the information stored in the guidance images. Extensive experimental results on several challenging datasets verify the effectiveness of our approach in achieving world consistency - the output video is consistent within the entire rendered 3D world. https://nvlabs.github.io/wc-vid2vid/

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 13; 2023: 17; 2024: 8; 2025: 8; 2026: 4.

69 papers.

Sort:

By area:

  1. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  2. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  3. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  4. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  5. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  6. 2021Taming Visually Guided Sound Generationcited by 136
  7. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  8. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  9. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  10. 2021GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worldscited by 98
  11. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 95
  12. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
38 more
  1. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  2. 2021Deepfake Video Detection Using Convolutional Vision Transformercited by 79
  3. 2021Learning to Stylize Novel Viewscited by 76
  4. 2021GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Drivingcited by 75
  5. 2023CityDreamer: Compositional Generative Model of Unbounded 3D Citiescited by 75
  6. 2021Enhancing Photorealism Enhancementcited by 74
  7. 2023SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collectionscited by 72
  8. 2021CCVS: Context-aware Controllable Video Synthesiscited by 68
  9. 2023InfiniCity: Infinite-Scale City Synthesiscited by 68
  10. 2022Signing at Scale: Learning to Co-Articulate Signs for Large-Scale Photo-Realistic Sign Language Productioncited by 61
  11. 2022Waymo Open Dataset: Panoramic Video Panoptic Segmentationcited by 57
  12. 2023Persistent Nature: A Generative Model of Unbounded 3D Worldscited by 44
  13. 2021iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesiscited by 42
  14. 2024A survey of synthetic data augmentation methods in computer visioncited by 34
  15. 2022Simple and Effective Synthesis of Indoor 3D Scenescited by 33
  16. 2021Dynamic Neural Garmentscited by 27
  17. 2024Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusioncited by 27
  18. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  19. 2023SignDiff: Diffusion Model for American Sign Language Productioncited by 25
  20. 2022NeurMiPs: Neural Mixture of Planar Experts for View Synthesiscited by 23
  21. 2021Long-Term Temporally Consistent Unpaired Video Translation from Simulated Surgical 3D Datacited by 21
  22. 2023TriVol: Point Cloud Rendering via Triple Volumescited by 16
  23. 2021Strumming to the Beat: Audio-Conditioned Contrastive Video Texturescited by 14
  24. 2021Diverse Generation from a Single Video Made Possiblecited by 13
  25. 2024From Efficient Multimodal Models to World Models: A Surveycited by 13
  26. 2025Dynamic View Synthesis as an Inverse Problemcited by 12
  27. 2022CompNVS: Novel View Synthesis with Scene Completioncited by 9
  28. 2024Anything in Any Scene: Photorealistic Video Object Insertioncited by 9
  29. 2024Planning from Imagination: Episodic Simulation and Episodic Memory for Vision-and-Language Navigationcited by 8
  30. 2021Semantics-aware Multi-modal Domain Translation:From LiDAR Point Clouds to Panoramic Color Imagescited by 6
  31. 2022SketchBetween: Video-to-Video Synthesis for Sprite Animation via Sketchescited by 6
  32. 2022Learning Temporally and Semantically Consistent Unpaired Video-to-video Translation Through Pseudo-Supervision From Synthetic Optical Flowcited by 5
  33. 2022Unsupervised High-Resolution Portrait Gaze Correction and Animationcited by 5
  34. 2025OmniView: An All-Seeing Diffusion Model for 3D and 4D View Synthesiscited by 4
  35. 2025Endless World: Real-Time 3D-Aware Long Video Generationcited by 4
  36. 2021JOKR: Joint Keypoint Representation for Unsupervised Cross-Domain Motion Retargetingcited by 3
  37. 2021Detection of GAN-synthesized street videoscited by 3
  38. 2025Video4Spatial: Towards Visuospatial Intelligence with Context-Guided Video Generationcited by 3

50 most cited of 69 All 69 (JSON)