# Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecture

arXiv: https://arxiv.org/abs/2403.07347 · HTML: https://mlsanity.com/paper/2403.07347/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 74 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 30 of 34)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [Learning A Sparse Transformer Network for Effective Image Deraining](https://mlsanity.com/paper/2303.11950/) (2023)
- [Audio-Visual Segmentation](https://mlsanity.com/paper/2207.05042/) (2022)
- [Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution](https://mlsanity.com/paper/1904.05049/) (2019)
- [Stripformer: Strip Transformer for Fast Image Deblurring](https://mlsanity.com/paper/2204.04627/) (2022)
- [Highly Accurate Dichotomous Image Segmentation](https://mlsanity.com/paper/2203.03041/) (2022)
- [Inception Transformer](https://mlsanity.com/paper/2205.12956/) (2022)
- [Omni-frequency Channel-selection Representations for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2203.00259/) (2022)
- [Fast Vision Transformers with HiLo Attention](https://mlsanity.com/paper/2205.13213/) (2022)
- [Contrastive Semi-supervised Learning for Underwater Image Restoration via Reliable Bank](https://mlsanity.com/paper/2303.09101/) (2023)
- [KVT: k-NN Attention for Boosting Vision Transformers](https://mlsanity.com/paper/2106.00515/) (2021)
- [A Study on ReLU and Softmax in Transformer](https://mlsanity.com/paper/2302.06461/) (2023)
- [Contrastive Positive Sample Propagation along the Audio-Visual Event Line](https://mlsanity.com/paper/2211.09980/) (2022)
- [Image Demoireing with Learnable Bandpass Filters](https://mlsanity.com/paper/2004.00406/) (2020)
- [Replacing softmax with ReLU in Vision Transformers](https://mlsanity.com/paper/2309.08586/) (2023)
- [Learning-based Video Motion Magnification](https://mlsanity.com/paper/1804.02684/) (2018)
- [Micron-BERT: BERT-based Facial Micro-Expression Recognition](https://mlsanity.com/paper/2304.03195/) (2023)
- [Revealing the Invisible with Model and Data Shrinking for Composite-database Micro-expression Recognition](https://mlsanity.com/paper/2006.09674/) (2020)
- [Adaptive Dynamic Filtering Network for Image Denoising](https://mlsanity.com/paper/2211.12051/) (2022)
- [EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformer](https://mlsanity.com/paper/2312.04152/) (2023)
- [Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers](https://mlsanity.com/paper/2205.14336/) (2022)
- [ViGT: Proposal-free Video Grounding with Learnable Token in Transformer](https://mlsanity.com/paper/2308.06009/) (2023)

## Built on by (most cited first, 28 of 28)

- [MMAD: Multi-label Micro-Action Detection in Videos](https://mlsanity.com/paper/2407.05311/) (2024)
- [Micro-gesture Online Recognition using Learnable Query Points](https://mlsanity.com/paper/2407.04490/) (2024)
- [Prototype Learning for Micro-gesture Classification](https://mlsanity.com/paper/2408.03097/) (2024)
- [MM-Gesture: Towards Precise Micro-Gesture Recognition through Multimodal Fusion](https://mlsanity.com/paper/2507.08344/) (2025)
- [Patch-level Sounding Object Tracking for Audio-Visual Question Answering](https://mlsanity.com/paper/2412.10749/) (2024)
- [Online Micro-gesture Recognition Using Data Augmentation and Spatial-Temporal Attention](https://mlsanity.com/paper/2507.09512/) (2025)
- [Motion Matters: Motion-guided Modulation Network for Skeleton-based Micro-Action Recognition](https://mlsanity.com/paper/2507.21977/) (2025)
- [Exploiting Ensemble Learning for Cross-View Isolated Sign Language Recognition](https://mlsanity.com/paper/2502.02196/) (2025)
- [XInsight: Integrative Stage-Consistent Psychological Counseling Support Agents for Digital Well-Being](https://mlsanity.com/paper/2603.06583/) (2026)
- [3D Smoke Scene Reconstruction Guided by Vision Priors from Multimodal Large Language Models](https://mlsanity.com/paper/2604.05687/) (2026)
- [Task-Generalized Adaptive Cross-Domain Learning for Multimodal Image Fusion](https://mlsanity.com/paper/2508.15505/) (2025)
- [Structure-guided Diffusion Transformer for Low-Light Image Enhancement](https://mlsanity.com/paper/2504.15054/) (2025)
- [Self-supervised Learning Matters: A Simple Ensemble Solution for Micro-Gesture Recognition](https://mlsanity.com/paper/2606.09261/) (2026)
- [FREAK: Frequency-modulated High-fidelity and Real-time Audio-driven Talking Portrait Synthesis](https://mlsanity.com/paper/2503.04067/) (2025)
- [Rethinking the Role of Feature Engineering and Learning Strategies in Few-Shot Hidden Emotion Recognition](https://mlsanity.com/paper/2606.31249/) (2026)
- [Prompt-Aware Controllable Shadow Removal](https://mlsanity.com/paper/2501.15043/) (2025)
- [Multimodal Protein Language Models for Enzyme Kinetic Parameters: From Substrate Recognition to Conformational Adaptation](https://mlsanity.com/paper/2603.12845/) (2026)
- [OLMD: Orientation-aware Long-term Motion Decoupling for Continuous Sign Language Recognition](https://mlsanity.com/paper/2503.08205/) (2025)
- [AMMSM: Adaptive Motion Magnification and Sparse Mamba for Micro-Expression Recognition](https://mlsanity.com/paper/2503.24057/) (2025)
- [Towards Micro-Action Recognition with Limited Annotations: An Asynchronous Pseudo Labeling and Training Approach](https://mlsanity.com/paper/2504.07785/) (2025)
- [MER-Bench: A Comprehensive Benchmark for Multimodal Meme Reappraisal](https://mlsanity.com/paper/2603.15020/) (2026)
- [FLAIR: Frequency- and Locality-Aware Implicit Neural Representations](https://mlsanity.com/paper/2508.13544/) (2025)
- [Training-Free Multimodal Deepfake Detection via Graph Reasoning](https://mlsanity.com/paper/2509.21774/) (2025)
- [GeoDiffMM: Geometry-Guided Conditional Diffusion for Motion Magnification](https://mlsanity.com/paper/2512.08325/) (2025)
- [Evidence Packing for Cross-Domain Image Deepfake Detection with LVLMs](https://mlsanity.com/paper/2603.17761/) (2026)
- [Beyond Shortcuts: Mitigating Visual Illusions in Frozen VLMs via Qualitative Reasoning](https://mlsanity.com/paper/2604.26250/) (2026)
- [GeoMag: Geometric-Aware Video Motion Magnification via State Space Model](https://mlsanity.com/paper/2605.29762/) (2026)
- [TextGaze: Prompting Gaze Target Estimation with Textual Scene Cues](https://mlsanity.com/paper/2607.10130/) (2026)

## Abstract (arXiv)

Video Motion Magnification (VMM) aims to reveal subtle and imperceptible motion information of objects in the macroscopic world. Prior methods directly model the motion field from the Eulerian perspective by Representation Learning that separates shape and texture or Multi-domain Learning from phase fluctuations. Inspired by the frequency spectrum, we observe that the low-frequency components with stable energy always possess spatial structure and less noise, making them suitable for modeling the subtle motion field. To this end, we present FD4MM, a new paradigm of Frequency Decoupling for Motion Magnification with a Multi-level Isomorphic Architecture to capture multi-level high-frequency details and a stable low-frequency structure (motion field) in video space. Since high-frequency details and subtle motions are susceptible to information degradation due to their inherent subtlety and unavoidable external interference from noise, we carefully design Sparse High/Low-pass Filters to enhance the integrity of details and motion structures, and a Sparse Frequency Mixer to promote seamless recoupling. Besides, we innovatively design a contrastive regularization for this task to strengthen the model's ability to discriminate irrelevant features, reducing undesired motion magnification. Extensive experiments on both Real-world and Synthetic Datasets show that our FD4MM outperforms SOTA methods. Meanwhile, FD4MM reduces FLOPs by 1.63× and boosts inference speed by 1.68× than the latest method. Our code is available at https://github.com/Jiafei127/FD4MM.