mlsanityData to 14 Oct 2026

NormalCrafter: Learning Temporally Consistent Normals from Video Diffusion Priors

arXiv 2504.11427 ↗First posted 15 Apr 2025cs.CVcited by 25

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Surface normal estimation serves as a cornerstone for a spectrum of computer vision applications. While numerous efforts have been devoted to static image scenarios, ensuring temporal coherence in video-based normal estimation remains a formidable challenge. Instead of merely augmenting existing methods with temporal components, we present NormalCrafter to leverage the inherent temporal priors of video diffusion models. To secure high-fidelity normal estimation across sequences, we propose Semantic Feature Regularization (SFR), which aligns diffusion features with semantic cues, encouraging the model to concentrate on the intrinsic semantics of the scene. Moreover, we introduce a two-stage training protocol that leverages both latent and pixel space learning to preserve spatial accuracy while maintaining long temporal context. Extensive evaluations demonstrate the efficacy of our method, showcasing a superior performance in generating temporally consistent normal sequences with intricate details from diverse videos.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 30 of its 40 references that are in the map. First posted on arXiv 15 Apr 2025.

Builds on

30 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think17 Sep
  2. 2024StableNormal: Reducing Diffusion Variance for Stable and Sharp Normal24 Jun
  3. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priors3 Jun
  4. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image18 Mar

Other references · 26

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  8. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  9. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  10. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  11. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  12. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
14 more
  1. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  2. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  3. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 204
  4. 2023MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyondcited by 185
  5. 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Predictioncited by 166
  6. 2018Evaluation of CNN-based Single-Image Depth Estimation Methodscited by 138
  7. 20193D Ken Burns Effect from a Single Imagecited by 128
  8. 2024Rethinking Inductive Biases for Surface Normal Estimationcited by 124
  9. 20223D Common Corruptions and Data Augmentationcited by 119
  10. 2021Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimationcited by 116
  11. 2019The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentationcited by 97
  12. 2024What Matters When Repurposing Diffusion Models for General Dense Perception Tasks?cited by 74
  13. 2020Surface Normal Estimation of Tilted Images via Spatial Rectifiercited by 34
  14. 2024Buffer Anytime: Zero-Shot Video Depth and Normal from Image Priorscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 18.

25 papers.

Sort:

By area:

  1. 2026Video Generation Models are General-Purpose Vision Learnerscited by 14
  2. 2025Lumen: Consistent Video Relighting and Harmonious Background Replacement with Video Generative Modelscited by 13
  3. 2025Diffusion Knows Transparency: Repurposing Video Diffusion for Transparent Object Depth and Normal Estimationcited by 10
  4. 2026Generative World Renderercited by 10
  5. 2026From Frames to Sequences: Temporally Consistent Human-Centric Dense Predictioncited by 9
  6. 2025CtrlVDiff: Controllable Video Generation via Unified Multimodal Video Diffusioncited by 6
  7. 2026THFM: A Unified Video Foundation Model for 4D Human Perception and Beyondcited by 5
  8. 2026Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generationcited by 5
  9. 2025MADrive: Memory-Augmented Driving Scene Modelingcited by 3
  10. 2026MMPhysVideo: Physically Plausible Video Generation Through Joint RGB-Perception Modelingcited by 3
  11. 2026UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priorscited by 3
  12. 2026Towards Consistent Video Geometry Estimationcited by 3
13 more
  1. 2026Monocular Normal Estimation via Shading Sequence Estimationcited by 2
  2. 2025Generative Perception of Shape and Material from Differential Motioncited by 1
  3. 2026Relit-LiVE: Relight Video by Jointly Learning Environment Videocited by 1
  4. 2026Real2SAM2Real: Generative 3D Caches as Complementary Context for Video Diffusioncited by 1
  5. 2026RealityBridge: Bridging Editable 3D Gaussian Splatting Driving Simulations and Real-World Videoscited by 1
  6. 2025Spatial Polarization Multiplexing: Single-Shot Invisible Shape and Reflectance Recoverycited by 0
  7. 2025TraceFlow: Dynamic 3D Reconstruction of Specular Scenes Driven by Ray Tracingcited by 0
  8. 2026LSA: Localized Semantic Alignment for Enhancing Temporal Consistency in Traffic Video Generationcited by 0
  9. 2026HumanOrbit: 3D Human Reconstruction as 360deg Orbit Generationcited by 0
  10. 2026FLAT: Feedforward Latent Triangle Splatting for Geometrically Accurate Scene Generationcited by 0
  11. 2026Unified Video Dense Prediction from Disjoint Datacited by 0
  12. 2026TransNormal-2: Geometry-Grounded Rectified Flow with Edge-Aware Decoding for Precise Normal Estimationcited by 0
  13. 2026AgentGarten: Code Worlds for Evolving Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Zero-Shot Video Semantic Segmentation based on Pre-Trained Diffusion Models
  2. 2025Fitting Image Diffusion Models on Video Datasets
  3. 2025InternVideo-Next: Towards General Video Foundation Models without Video-Text Supervision
  4. 2025UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation
  5. 2026ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning
  6. 2026DVD: Deterministic Video Depth Estimation with Generative Priors
  7. 2024Exploring Pre-trained Text-to-Video Diffusion Models for Referring Video Object Segmentation
  8. 2025SpatialCrafter: Unleashing the Imagination of Video Diffusion Models for Scene Reconstruction from Limited Observations