Phi-Noise: Training-Free Temporal Video Conditioning via Phase-Based Noise Manipulation
On the map: Diffusion models → Video generation
Abstract (arXiv)
Latent video diffusion models generate videos by progressively transforming Gaussian noise into realistic samples conditioned on text or visual inputs. However, existing conditioning methods often require additional training and computational overhead. Motivated by recent findings on the importance of frequency components in generative models, we propose a simple, training-free approach for motion-conditioned video generation by injecting low-frequency phase information from a reference video directly into the diffusion noise latents. Our method transfers motion cues without modifying the model architecture or inference pipeline. Using several applications, we demonstrate effective control over both appearance and dynamics in generated videos, while achieving competitive or superior results compared to more complex conditioning approaches.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 48 of its 54 references that are in the map. First posted on arXiv 23 May 2026.
Builds on
48 of 54 references.
Closest ancestors · 17
Same direction, newest first
- 2025Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising9 Nov
- 2025FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise5 Feb
- 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noise14 Jan
- 2024Video Motion Transfer with Diffusion Transformers10 Dec
- 2024OnlyFlow: Optical Flow based Motion Conditioning for Video Diffusion Models15 Nov
- 2024Video Diffusion Models are Training-free Motion Interpreter and Controller23 May
- 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion5 Feb
- 2024Lumiere: A Space-Time Diffusion Model for Video Generation23 Jan
- 2023FreeInit: Bridging Initialization Gap in Video Diffusion Models12 Dec
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation28 Nov
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
- 2023VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNet26 Jul
5 more
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
- 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators23 Mar
- 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
- 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov
Other references · 31
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
- 2025Qwen-Image Technical Reportcited by 1,044
- 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
- 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
19 more
- 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 294
- 2023Style Aligned Image Generation via Shared Attentioncited by 216
- 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 195
- 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 178
- 2022Generative Modelling With Inverse Heat Dissipationcited by 167
- 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
- 2024MotionClone: Training-Free Motion Cloning for Controllable Video Generationcited by 105
- 2023DragVideo: Interactive Drag-style Video Editingcited by 40
- 2025How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Modelscited by 32
- 2024Motion Inversion for Video Customizationcited by 30
- 2024Blue noise for diffusion modelscited by 29
- 2024DiffUHaul: A Training-Free Method for Object Dragging in Imagescited by 22
- 2022Score-based Denoising Diffusion with Non-Isotropic Gaussian Noise Modelscited by 17
- 2025FDS: Frequency-Aware Denoising Score for Text-Guided Latent Diffusion Image Editingcited by 12
- 2024Conditional Balance: Improving Multi-Conditioning Trade-Offs in Image Generationcited by 9
- 2024Infinite-Resolution Integral Noise Warping for Diffusion Modelscited by 5
- 2025TempoControl: Temporal Attention Guidance for Text-to-Video Modelscited by 5
- 2025NeuralRemaster: Phase-Preserving Diffusion for Structure-Aligned Generationcited by 5
- 2026Colorful-Noise: Training-Free Low-Frequency Noise Manipulation for Color-Based Conditional Image Generationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2022Video Diffusion Models
- 2024Noise Crystallization and Liquid Noise: Zero-shot Video Generation using Image Diffusion Models
- 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models
- 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling
- 2026Equilibrium Forcing: Adaptive Video Generation Without Noise Conditioning
- 2023APLA: Additional Perturbation for Latent Noise with Adversarial Training Enables Consistency
- 2026Test-Time Noise Guided Adaptation for Realistic Autoregressive Video Generation
- 2024TVG: A Training-free Transition Video Generation Method with Diffusion Models