EasyVFX: Frequency-Driven Decoupling for Resource-Efficient VFX Generation
On the map: Mixture-of-experts → Vision mixture-of-experts
Abstract (arXiv)
Generating high-fidelity visual effects (VFX) typically demands massive datasets and prohibitive computational power due to the intricate coupling of spatial textures and temporal dynamics. In this paper, we introduce EasyVFX, a resource-efficient framework that achieves realistic VFX synthesis under stringent constraints. Our core philosophy lies in frequency-domain decomposition: we observe that the complexity of VFX can be significantly mitigated by decoupling high-frequency components, which represent intricate spatial appearances, from low-frequency components that encapsulate global motion dynamics. This spectral disentanglement transforms a high-dimensional learning problem into manageable sub-tasks, thereby lowering the optimization barrier and reducing data dependency. Building upon this insight, we propose a two-stage training paradigm. First, we design a Frequency-aware Mixture-of-Experts (Freq-MoE) architecture. By utilizing a soft routing mechanism, our model assigns specialized experts to distinct spectral bands, enabling them to cultivate robust priors for appearance and motion dynamics. This specialization allows the model to acquire foundational VFX knowledge with fewer GPU resources. Second, we introduce a Test-Time Training strategy powered by a novel Frequency-constraint Loss. This allows the pre-trained model to swiftly adapt to specific, unseen effects through localized optimizations, requiring only about 100 steps on a single GPU. Experimental results demonstrate that EasyVFX produces structurally consistent and visually stunning effects, proving that frequency-aware learning is a key catalyst for democratizing professional-grade VFX.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 64 of its 79 references that are in the map. First posted on arXiv 21 May 2026.
Builds on
64 of 79 references.
References · 64
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
- 2025VACE: All-in-One Video Creation and Editingcited by 479
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
- 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
38 more
- 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
- 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 272
- 2024TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methodscited by 249
- 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 206
- 2024Taming Rectified Flow for Inversion and Editingcited by 204
- 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 188
- 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 178
- 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 175
- 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 174
- 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
- 2024ControlNeXt: Powerful and Efficient Control for Image and Video Generationcited by 159
- 2023SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generationcited by 143
- 2023Make-Your-Video: Customized Video Generation Using Textual and Structural Guidancecited by 129
- 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 119
- 2024DUET: Dual Clustering Enhanced Multivariate Time Series Forecastingcited by 107
- 2024MotionClone: Training-Free Motion Cloning for Controllable Video Generationcited by 105
- 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 102
- 2024DiT4Edit: Diffusion Transformer for Image Editingcited by 101
- 2025Controllable Video Generation: A Surveycited by 98
- 2025Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuningcited by 81
- 2024CustomVideo: Customizing Text-to-Video Generation with Multiple Subjectscited by 77
- 2025SkyReels-A2: Compose Anything in Video Diffusion Transformerscited by 73
- 2025Follow-Your-Creation: Empowering 4D Creation through Video Inpaintingcited by 62
- 2025MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generationcited by 45
- 2025EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editingcited by 45
- 2025DBLoss: Decomposition-based Loss Function for Time Series Forecastingcited by 45
- 2024Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Modelscited by 44
- 2025LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformercited by 43
- 2025Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animationcited by 43
- 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlookcited by 43
- 2025DAG: A Dual Correlation Network for Time Series Forecasting with Exogenous Variablescited by 33
- 2025MotionPro: A Precise Motion Controller for Image-to-Video Generationcited by 30
- 2025Follow-Your-Shape: Shape-Aware Image Editing via Trajectory-Guided Region Controlcited by 28
- 2025Omni-Effects: Unified and Spatially-Controllable Visual Effects Generationcited by 27
- 2025FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarioscited by 25
- 2026FastVMT: Eliminating Redundancy in Video Motion Transfercited by 23
- 2025Through-The-Mask: Mask-based Motion Trajectories for Image-to-Video Generationcited by 20
- 2025Follow-Your-Preference: Towards Preference-Aligned Image Inpaintingcited by 18
50 most cited of 64 All 64 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Toward Diffusible High-Dimensional Latent Spaces: A Frequency Perspective
- 2025Decoupling High and Low Frequencies for Faithful Image Generation with Fine Details
- 2023Generative Image Dynamics
- 2026FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generation
- 2026Diff-VF: Training-free High-quality Long Video Generation via Diffusion Model
- 2025Physics-Guided Motion Loss for Video Generation Model
- 2024FreeScale: Unleashing the Resolution of Diffusion Models via Tuning-Free Scale Fusion
- 2025PromptVFX: Text-Driven Fields for Open-World 3D Gaussian Animation