# Delving into the Frequency: Temporally Consistent Human Motion Transfer in the Fourier Space arXiv: https://arxiv.org/abs/2209.00233 ยท HTML: https://mlsanity.com/paper/2209.00233/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 52 references that are in the map. - First posted on arXiv 1 Sep 2022. ## Builds on (most cited first, 30 of 30) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016) - [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020) - [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018) - [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016) - [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021) - [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017) - [Leveraging Frequency Analysis for Deep Fake Image Recognition](https://mlsanity.com/paper/2003.08685/) (2020) - [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018) - [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018) - [Detecting and Simulating Artifacts in GAN Fake Images](https://mlsanity.com/paper/1907.06515/) (2019) - [Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions](https://mlsanity.com/paper/2003.01826/) (2020) - [Learning Blind Video Temporal Consistency](https://mlsanity.com/paper/1808.00449/) (2018) - [Neural Voice Puppetry: Audio-driven Facial Reenactment](https://mlsanity.com/paper/1912.05566/) (2019) - [Animating Arbitrary Objects via Deep Motion Transfer](https://mlsanity.com/paper/1812.08861/) (2018) - [Few-shot Video-to-Video Synthesis](https://mlsanity.com/paper/1910.12713/) (2019) - [Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis](https://mlsanity.com/paper/1909.12224/) (2019) - [Putting People in their Place: Monocular Regression of 3D People in Depth](https://mlsanity.com/paper/2112.08274/) (2021) - [Fourier Spectrum Discrepancies in Deep Network Generated Images](https://mlsanity.com/paper/1911.06465/) (2019) - [Gait Recognition in the Wild with Dense 3D Representations and A Benchmark](https://mlsanity.com/paper/2204.02569/) (2022) - [Synthesizing Images of Humans in Unseen Poses](https://mlsanity.com/paper/1804.07739/) (2018) - [Blind Video Temporal Consistency via Deep Video Prior](https://mlsanity.com/paper/2010.11838/) (2020) - [Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis](https://mlsanity.com/paper/1810.11610/) (2018) - [Recent Advances in Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspective](https://mlsanity.com/paper/2104.11536/) (2021) - [SSD-GAN: Measuring the Realness in the Spatial and Spectral Domains](https://mlsanity.com/paper/2012.05535/) (2020) - [Liquid Warping GAN with Attention: A Unified Framework for Human Image Synthesis](https://mlsanity.com/paper/2011.09055/) (2020) - [GAC-GAN: A General Method for Appearance-Controllable Human Video Motion Transfer](https://mlsanity.com/paper/1911.10672/) (2019) - [C2F-FWN: Coarse-to-Fine Flow Warping Network for Spatial-Temporal Consistent Motion Transfer](https://mlsanity.com/paper/2012.08976/) (2020) ## Built on by (most cited first, 5 of 5) - [Make-Your-Anchor: A Diffusion-based 2D Avatar Generation Framework](https://mlsanity.com/paper/2403.16510/) (2024) - [Spectral Motion Alignment for Video Motion Transfer using Diffusion Models](https://mlsanity.com/paper/2403.15249/) (2024) - [FREAK: Frequency-modulated High-fidelity and Real-time Audio-driven Talking Portrait Synthesis](https://mlsanity.com/paper/2503.04067/) (2025) - [SRENet: Spectral Re-Entry Network for Point Cloud Action Recognition](https://mlsanity.com/paper/2606.03160/) (2026) - [DISPLAY: Directable Human-Object Interaction Video Generation via Sparse Motion Guidance and Multi-Task Auxiliary](https://mlsanity.com/paper/2603.09883/) (2026) ## Abstract (arXiv) Human motion transfer refers to synthesizing photo-realistic and temporally coherent videos that enable one person to imitate the motion of others. However, current synthetic videos suffer from the temporal inconsistency in sequential frames that significantly degrades the video quality, yet is far from solved by existing methods in the pixel domain. Recently, some works on DeepFake detection try to distinguish the natural and synthetic images in the frequency domain because of the frequency insufficiency of image synthesizing methods. Nonetheless, there is no work to study the temporal inconsistency of synthetic videos from the aspects of the frequency-domain gap between natural and synthetic videos. In this paper, we propose to delve into the frequency space for temporally consistent human motion transfer. First of all, we make the first comprehensive analysis of natural and synthetic videos in the frequency domain to reveal the frequency gap in both the spatial dimension of individual frames and the temporal dimension of the video. To close the frequency gap between the natural and synthetic videos, we propose a novel Frequency-based human MOtion TRansfer framework, named FreMOTR, which can effectively mitigate the spatial artifacts and the temporal inconsistency of the synthesized videos. FreMOTR explores two novel frequency-based regularization modules: 1) the Frequency-domain Appearance Regularization (FAR) to improve the appearance of the person in individual frames and 2) Temporal Frequency Regularization (TFR) to guarantee the temporal consistency between adjacent frames. Finally, comprehensive experiments demonstrate that the FreMOTR not only yields superior performance in temporal consistency metrics but also improves the frame-level visual quality of synthetic videos. In particular, the temporal consistency metrics are improved by nearly 30% than the state-of-the-art model.