# NeMF: Neural Motion Fields for Kinematic Animation

arXiv: https://arxiv.org/abs/2206.03287 · HTML: https://mlsanity.com/paper/2206.03287/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 83 references that are in the map.
- First posted on arXiv 4 Jun 2022.

## Builds on (most cited first, 30 of 33)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes](https://mlsanity.com/paper/2104.03953/) (2021)
- [Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing](https://mlsanity.com/paper/1903.10145/) (2019)
- [DLow: Diversifying Latent Flows for Diverse Human Motion Prediction](https://mlsanity.com/paper/2003.08386/) (2020)
- [Character Controllers Using Motion VAEs](https://mlsanity.com/paper/2103.14274/) (2021)
- [History Repeats Itself: Human Motion Prediction via Motion Attention](https://mlsanity.com/paper/2007.11755/) (2020)
- [A Spatio-temporal Transformer for 3D Human Motion Prediction](https://mlsanity.com/paper/2004.08692/) (2020)
- [Skeleton-Aware Networks for Deep Motion Retargeting](https://mlsanity.com/paper/2005.05732/) (2020)
- [We are More than Our Joints: Predicting how 3D Bodies Move](https://mlsanity.com/paper/2012.00619/) (2020)
- [QuaterNet: A Quaternion-based Recurrent Model for Human Motion](https://mlsanity.com/paper/1805.06485/) (2018)
- [LEAP: Learning Articulated Occupancy of People](https://mlsanity.com/paper/2104.06849/) (2021)
- [Learning Motion Priors for 4D Human Body Capture in 3D Scenes](https://mlsanity.com/paper/2108.10399/) (2021)
- [Neural Kinematic Networks for Unsupervised Motion Retargetting](https://mlsanity.com/paper/1804.05653/) (2018)
- [Learning to Generate Diverse Dance Motions with Transformer](https://mlsanity.com/paper/2008.08171/) (2020)
- [Convolutional Autoencoders for Human Motion Infilling](https://mlsanity.com/paper/2010.11531/) (2020)
- [COAP: Compositional Articulated Occupancy of People](https://mlsanity.com/paper/2204.06184/) (2022)

## Built on by (most cited first, 30 of 68)

- [PhysDiff: Physics-Guided Human Motion Diffusion Model](https://mlsanity.com/paper/2212.02500/) (2022)
- [EDGE: Editable Dance Generation From Music](https://mlsanity.com/paper/2211.10658/) (2022)
- [Object Motion Guided Human Motion Synthesis](https://mlsanity.com/paper/2309.16237/) (2023)
- [Controllable Human-Object Interaction Synthesis](https://mlsanity.com/paper/2312.03913/) (2023)
- [TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videos](https://mlsanity.com/paper/2403.17346/) (2024)
- [Ego-Body Pose Estimation via Ego-Head Pose Estimation](https://mlsanity.com/paper/2212.04636/) (2022)
- [Flexible Motion In-betweening with Diffusion Models](https://mlsanity.com/paper/2405.11126/) (2024)
- [Single Motion Diffusion](https://mlsanity.com/paper/2302.05905/) (2023)
- [Estimating Body and Hand Motion in an Ego-sensed World](https://mlsanity.com/paper/2410.03665/) (2024)
- [ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactions](https://mlsanity.com/paper/2401.10232/) (2024)
- [Autonomous Character-Scene Interaction Synthesis from Text Instruction](https://mlsanity.com/paper/2410.03187/) (2024)
- [Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Camera](https://mlsanity.com/paper/2412.12861/) (2024)
- [Generating Human Motion in 3D Scenes from Text Descriptions](https://mlsanity.com/paper/2405.07784/) (2024)
- [GENMO: A GENeralist Model for Human MOtion](https://mlsanity.com/paper/2505.01425/) (2025)
- [Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Models](https://mlsanity.com/paper/2310.02242/) (2023)
- [Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations](https://mlsanity.com/paper/2303.13129/) (2023)
- [Generating Continual Human Motion in Diverse 3D Scenes](https://mlsanity.com/paper/2304.02061/) (2023)
- [PACE: Human and Camera Motion Estimation from in-the-wild Videos](https://mlsanity.com/paper/2310.13768/) (2023)
- [TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolation](https://mlsanity.com/paper/2410.04221/) (2024)
- [ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generation](https://mlsanity.com/paper/2412.18600/) (2024)
- [HMP: Hand Motion Priors for Pose and Shape Estimation from Video](https://mlsanity.com/paper/2312.16737/) (2023)
- [HUMOS: Human Motion Model Conditioned on Body Shape](https://mlsanity.com/paper/2409.03944/) (2024)
- [RSMT: Real-time Stylized Motion Transition for Characters](https://mlsanity.com/paper/2306.11970/) (2023)
- [HOIDiNi: Human-Object Interaction through Diffusion Noise Optimization](https://mlsanity.com/paper/2506.15625/) (2025)
- [Synergistic Global-space Camera and Human Reconstruction from Videos](https://mlsanity.com/paper/2405.14855/) (2024)
- [UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction, Forecasting, and Generation](https://mlsanity.com/paper/2508.01126/) (2025)
- [Example-based Motion Synthesis via Generative Motion Matching](https://mlsanity.com/paper/2306.00378/) (2023)
- [SemGeoMo: Dynamic Contextual Human Motion Generation with Semantic and Geometric Guidance](https://mlsanity.com/paper/2503.01291/) (2025)
- [COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation](https://mlsanity.com/paper/2408.16426/) (2024)
- [Diffgrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model](https://mlsanity.com/paper/2412.20657/) (2024)

## Abstract (arXiv)

We present an implicit neural representation to learn the spatio-temporal space of kinematic motions. Unlike previous work that represents motion as discrete sequential samples, we propose to express the vast motion space as a continuous function over time, hence the name Neural Motion Fields (NeMF). Specifically, we use a neural network to learn this function for miscellaneous sets of motions, which is designed to be a generative model conditioned on a temporal coordinate t and a random vector z for controlling the style. The model is then trained as a Variational Autoencoder (VAE) with motion encoders to sample the latent space. We train our model with a diverse human motion dataset and quadruped dataset to prove its versatility, and finally deploy it as a generic motion prior to solve task-agnostic problems and show its superiority in different motion generation and editing applications, such as motion interpolation, in-betweening, and re-navigating. More details can be found on our project page: https://cs.yale.edu/homes/che/projects/nemf/.