mlsanityData to 7 Oct 2026

How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Models

arXiv 2504.03072 ↗First posted 3 Apr 2025cs.CV cs.LGcited by 31

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Video editing and generation methods often rely on pre-trained image-based diffusion models. During the diffusion process, however, the reliance on rudimentary noise sampling techniques that do not preserve correlations present in subsequent frames of a video is detrimental to the quality of the results. This either produces high-frequency flickering, or texture-sticking artifacts that are not amenable to post-processing. With this in mind, we propose a novel method for preserving temporal correlations in a sequence of noise samples. This approach is materialized by a novel noise representation, dubbed int-noise (integral noise), that reinterprets individual noise samples as a continuously integrated noise field: pixel values do not represent discrete values, but are rather the integral of an underlying infinite-resolution noise over the pixel area. Additionally, we propose a carefully tailored transport method that uses int-noise to accurately advect noise samples over a sequence of frames, maximizing the correlation between different frames while also preserving the noise properties. Our results demonstrate that the proposed int-noise can be used for a variety of tasks, such as video restoration, surrogate rendering, and conditional video generation. See https://warpyournoise.github.io/ for video results.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 47 of its 60 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

47 of 60 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance11 Oct
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  3. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  4. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  6. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 41

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  6. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  7. 2022Video Diffusion Modelscited by 2,280
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  9. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  10. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  11. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  12. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
29 more
  1. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  2. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  3. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  4. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  5. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  6. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  7. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  8. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  9. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  10. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
  11. 2023Zero-shot Image-to-Image Translationcited by 503
  12. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 501
  13. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  14. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  15. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  16. 2022Inversion-Based Style Transfer with Diffusion Modelscited by 290
  17. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  18. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 256
  19. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  20. 2018Learning Blind Video Temporal Consistencycited by 247
  21. 2023An Edit Friendly DDPM Noise Space: Inversion and Manipulationscited by 243
  22. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editingcited by 182
  23. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  24. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  25. 2022Person Image Synthesis via Denoising Diffusion Modelcited by 104
  26. 2023Edit-A-Video: Single Video Editing with Object-Aware Consistencycited by 65
  27. 2023Exact Diffusion Inversion via Bi-directional Integration Approximationcited by 43
  28. 2023MeDM: Mediating Image Diffusion Models for Video-to-Video Translation with Temporal Correspondence Guidancecited by 17
  29. 2023DIFF-NST: Diffusion Interleaving For deFormable Neural Style Transfercited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 19.

31 papers.

Sort:

By area:

  1. 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 40
  2. 2025Time Blindness: Why Video-Language Models Can't See What Humans Can?cited by 17
  3. 2025Elucidated Rolling Diffusion Models for Probabilistic Forecasting of Complex Dynamicscited by 12
  4. 2025BulletTime: Decoupled Control of Time and Camera Pose for Video Generationcited by 12
  5. 2025LookingGlass: Generative Anamorphoses via Laplacian Pyramid Warpingcited by 7
  6. 2025Taming Flow-based I2V Models for Creative Video Editingcited by 5
  7. 2025On Equivariance and Fast Sampling in Video Diffusion Models Trained with Warped Noisecited by 4
  8. 2025AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Pathcited by 4
  9. 2025STeP: A Framework for Solving Scientific Video Inverse Problems with Spatiotemporal Diffusion Priorscited by 3
  10. 2025InstantViR: Real-Time Video Inverse Problem Solver with Distilled Diffusion Priorcited by 3
  11. 2026CameraNoise: Enabling Faithful Camera Control in Video Diffusion through Geometry-Flow-Guided Noise Warpingcited by 3
  12. 2026Inference-Time Scaling for Joint Audio-Video Generationcited by 3
19 more
  1. 2026Preference Score Distillation: Leveraging 2D Rewards to Align Text-to-3D Generation with Human Preferencecited by 2
  2. 2025Time-Correlated Video Bridge Matchingcited by 1
  3. 2026Variational Trajectory Optimization of Anisotropic Diffusion Schedulescited by 1
  4. 2026Generative Phomosaic with Structure-Aligned and Personalized Diffusioncited by 1
  5. 2026NeoMap: Training-free Novel-View Synthesis from Single Images and Videoscited by 1
  6. 2026Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generationcited by 1
  7. 2025PS4PRO: Pixel-to-pixel Supervision for Photorealistic Rendering and Optimizationcited by 0
  8. 2026Region-Adaptive Generative Compression with Spatially Varying Diffusion Modelscited by 0
  9. 2026HumANDiff: Articulated Noise Diffusion for Motion-Consistent Human Video Generationcited by 0
  10. 2026FlowPalm: Optical Flow Driven Non-Rigid Deformation for Geometrically Diverse Palmprint Generationcited by 0
  11. 2026Accelerating Video Inverse Problem Solvers with Autoregressive Diffusion Modelscited by 0
  12. 2026CoMoGen: COntrollable MOtion Dynamics and Interactions with Mask-Guided Video GENerationcited by 0
  13. 2026Phi-Noise: Training-Free Temporal Video Conditioning via Phase-Based Noise Manipulationcited by 0
  14. 2026SpecLoR: Spectral Lookahead Rectification for Motion-Coherent Text-to-Video Generationcited by 0
  15. 20264D Human-Scene Reconstruction from Low-Overlap Capturescited by 0
  16. 2026JoLT: Joint Latent Trajectories for Context-Guided High-Resolution Tiled Generationcited by 0
  17. 2026Spatially-Grounded Flow Matching: Structured Source Distributions for Image Generationcited by 0
  18. 2026SceneHI: High-Resolution 3D-Consistent Scene Texturing with Controllable Illuminationcited by 0
  19. 2026NowcastDiT: Diffusion Transformers are Effective Precipitation Nowcasterscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Infinite-Resolution Integral Noise Warping for Diffusion Models
  2. 2024Blue noise for diffusion models
  3. 2024Noise Calibration: Plug-and-play Content-Preserving Video Enhancement using Pre-trained Video Diffusion Models
  4. 2025CARD: Correlation Aware Restoration with Diffusion
  5. 2026Spectrally-Guided Diffusion Noise Schedules
  6. 2024NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation
  7. 2025FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise
  8. 2024Tuning-Free Noise Rectification for High Fidelity Image-to-Video Generation