mlsanityData to 13 Oct 2026

BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction

arXiv 2211.14304 ↗First posted 25 Nov 2022cs.CVcited by 82

On the map: Diffusion models → Video generation

Abstract (arXiv)

Stochastic human motion prediction (HMP) has generally been tackled with generative adversarial networks and variational autoencoders. Most prior works aim at predicting highly diverse movements in terms of the skeleton joints' dispersion. This has led to methods predicting fast and motion-divergent movements, which are often unrealistic and incoherent with past motion. Such methods also neglect contexts that need to anticipate diverse low-range behaviors, or actions, with subtle joint displacements. To address these issues, we present BeLFusion, a model that, for the first time, leverages latent diffusion models in HMP to sample from a latent space where behavior is disentangled from pose and motion. As a result, diversity is encouraged from a behavioral perspective. Thanks to our behavior coupler's ability to transfer sampled behavior to ongoing motion, BeLFusion's predictions display a variety of behaviors that are significantly more realistic than the state of the art. To support it, we introduce two metrics, the Area of the Cumulative Motion Distribution, and the Average Pairwise Distance Error, which are correlated to our definition of realism according to a qualitative study with 126 participants. Finally, we prove BeLFusion's generalization power in a new cross-dataset scenario for stochastic HMP.

Key facts. Cited by 82 papers in the map (data through 13 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 60 of its 83 references that are in the map. First posted on arXiv 25 Nov 2022.

Builds on

60 of 83 references.

References · 60

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  4. 2020Denoising Diffusion Implicit Modelscited by 9,718
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,213
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,023
  7. 2022Generative Adversarial Networkscited by 5,614
  8. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,306
  9. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,213
  10. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,112
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,012
  12. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 976
38 more
  1. 2019On the Convergence of Adam and Beyondcited by 837
  2. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 688
  3. 2021Score-based Generative Modeling in Latent Spacecited by 635
  4. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 579
  5. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 516
  6. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 498
  7. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 438
  8. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 398
  9. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  10. 2022Understanding Diffusion Models: A Unified Perspectivecited by 340
  11. 2017On human motion prediction using recurrent neural networkscited by 337
  12. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecastingcited by 335
  13. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noisecited by 275
  14. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 245
  15. 2015Structural-RNN: Deep Learning on Spatio-Temporal Graphscited by 234
  16. 2019Human Motion Trajectory Prediction: A Surveycited by 231
  17. 2016Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoderscited by 219
  18. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 185
  19. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 184
  20. 2017HP-GAN: Probabilistic 3D human motion prediction via GANcited by 176
  21. 2020History Repeats Itself: Human Motion Prediction via Motion Attentioncited by 164
  22. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  23. 2020A Spatio-temporal Transformer for 3D Human Motion Predictioncited by 133
  24. 2020Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Predictioncited by 132
  25. 2018Convolutional Sequence to Sequence Model for Human Dynamicscited by 115
  26. 2018QuaterNet: A Quaternion-based Recurrent Model for Human Motioncited by 108
  27. 2021MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Predictioncited by 100
  28. 2022Back to MLP: A Simple Baseline for Human Motion Predictioncited by 99
  29. 2017The Pose Knows: Video Forecasting by Generating Pose Futurescited by 88
  30. 2018MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamicscited by 86
  31. 2019Diverse Trajectory Forecasting with Determinantal Point Processescited by 80
  32. 2017DeLiGAN : Generative Adversarial Networks for Diverse and Limited Datacited by 77
  33. 2021Generating Smooth Pose Sequences for Diverse Human Motion Predictioncited by 66
  34. 2018Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objectivecited by 56
  35. 2018BiHMP-GAN: Bidirectional 3D Human Motion Prediction GANcited by 55
  36. 2021Pose Transformers (POTR): Human Motion Prediction with Non-Autoregressive Transformerscited by 46
  37. 2022A generic diffusion-based approach for 3D human pose prediction in the wildcited by 33
  38. 2022Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an Auxiliary Spacecited by 32

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 15; 2024: 31; 2025: 21; 2026: 15.

82 papers.

Sort:

By area:

  1. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  2. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 138
  3. 2023Controllable Human-Object Interaction Synthesiscited by 108
  4. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 91
  5. 2023HumanMAC: Masked Motion Completion for Human Motion Predictioncited by 81
  6. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 79
  7. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  8. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  9. 2024ReGenNet: Towards Human Action-Reaction Synthesiscited by 53
  10. 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
  11. 2024RoHM: Robust Human Motion Reconstruction via Diffusioncited by 47
  12. 2023Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Modelscited by 43
38 more
  1. 2024Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioningcited by 29
  2. 2024in2IN: Leveraging individual Information to Generate Human INteractionscited by 29
  3. 2024ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generationcited by 25
  4. 2023CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusioncited by 21
  5. 2023Dance with You: The Diversity Controllable Dancer Generation via Diffusion Modelscited by 17
  6. 2025UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction, Forecasting, and Generationcited by 17
  7. 2023TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Predictioncited by 16
  8. 2024REACT 2024: the Second Multiple Appropriate Facial Reaction Generation Challengecited by 16
  9. 2024Learning Semantic Latent Directions for Accurate and Controllable Human Motion Predictioncited by 14
  10. 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directionscited by 14
  11. 2025MixerMDM: Learnable Composition of Human Motion Diffusion Modelscited by 13
  12. 2025Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Predictioncited by 12
  13. 2025From Sparse Signal to Smooth Motion: Real-Time Motion Generation with Rolling Prediction Modelscited by 11
  14. 2025MOSPA: Human Motion Generation Driven by Spatial Audiocited by 11
  15. 2024Local Action-Guided Motion Diffusion Model for Text-to-Motion Generationcited by 10
  16. 2023GazeMoDiff: Gaze-guided Diffusion Model for Stochastic Human Motion Predictioncited by 9
  17. 2024MATRIX: Multi-Agent Trajectory Generation with Diverse Contextscited by 9
  18. 2024Motion-2-To-3: Leveraging 2D Motion Data for 3D Motion Generationscited by 9
  19. 2024Text-guided 3D Human Motion Generation with Keyframe-based Parallel Skip Transformercited by 8
  20. 2024MotionWavelet: Human Motion Prediction via Wavelet Manifold Learningcited by 8
  21. 2025Motion Generation: A Survey of Generative Approaches and Benchmarkscited by 8
  22. 2024FORCE: Physics-aware Human-object Interactioncited by 7
  23. 2024MotionMap: Representing Multimodality in Human Pose Forecastingcited by 7
  24. 2025EgoReAct: Egocentric Video-Driven 3D Human Reaction Generationcited by 7
  25. 2024Gated Temporal Diffusion for Stochastic Long-Term Dense Anticipationcited by 6
  26. 2024DivDiff: A Conditional Diffusion Model for Diverse Human Motion Predictioncited by 6
  27. 2023REACT2023: the first Multi-modal Multiple Appropriate Facial Reaction Generation Challengecited by 5
  28. 2026Unleashing Guidance Without Classifiers for Human-Object Interaction Animationcited by 5
  29. 2024CrowdMoGen: Zero-Shot Text-Driven Collective Motion Generationcited by 4
  30. 2024Arbitrarily-Conditioned Multi-Functional Diffusion for Multi-Physics Emulationcited by 4
  31. 2025CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flowcited by 4
  32. 2023Controllable Motion Synthesis and Reconstruction with Autoregressive Diffusion Modelscited by 3
  33. 2024Controllable Diverse Sampling for Diffusion Based Motion Behavior Forecastingcited by 3
  34. 2024Text to Blind Motioncited by 3
  35. 2025Latent Behavior Diffusion for Sequential Reaction Generation in Dyadic Settingcited by 3
  36. 2023MRecGen: Multimodal Appropriate Reaction Generatorcited by 2
  37. 2023Diffusion-Based Particle-DETR for BEV Perceptioncited by 2
  38. 2024Massively Multi-Person 3D Human Motion Forecasting with Scene Contextcited by 2

50 most cited of 82 All 82 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Unifying Human Motion Synthesis and Style Transfer with Denoising Diffusion Probabilistic Models
  2. 2026EquiFusion: Kinematics-Agnostic Human Motion Prediction via Equivariant Latent Diffusion
  3. 2025Back to Basics: Motion Representation Matters for Human Motion Generation Using Diffusion Model
  4. 2022Pretrained Diffusion Models for Unified Human Motion Synthesis
  5. 2022Executing your Commands via Motion Diffusion in Latent Space
  6. 2024Predicting Long-Term Human Behaviors in Discrete Representations via Physics-Guided Diffusion
  7. 2024Length-Aware Motion Synthesis via Latent Diffusion
  8. 2024Multi-Condition Latent Diffusion Network for Scene-Aware Neural Human Motion Prediction