# Bayesian-Optimized One-Step Diffusion Model with Knowledge Distillation for Real-Time 3D Human Motion Prediction

arXiv: https://arxiv.org/abs/2409.12456 · HTML: https://mlsanity.com/paper/2409.12456/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 23 of its 31 references that are in the map.
- First posted on arXiv 19 Sep 2024.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 23 of 23)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation](https://mlsanity.com/paper/2309.06380/) (2023)
- [Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed](https://mlsanity.com/paper/2101.02388/) (2021)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [DLow: Diversifying Latent Flows for Diverse Human Motion Prediction](https://mlsanity.com/paper/2003.08386/) (2020)
- [HP-GAN: Probabilistic 3D human motion prediction via GAN](https://mlsanity.com/paper/1711.09561/) (2017)
- [Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation](https://mlsanity.com/paper/2105.08919/) (2021)
- [MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics](https://mlsanity.com/paper/1808.04545/) (2018)
- [BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction](https://mlsanity.com/paper/2211.14304/) (2022)
- [HumanMAC: Masked Motion Completion for Human Motion Prediction](https://mlsanity.com/paper/2302.03665/) (2023)
- [DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data](https://mlsanity.com/paper/1706.02071/) (2017)
- [Human Joint Kinematics Diffusion-Refinement for Stochastic Motion Prediction](https://mlsanity.com/paper/2210.05976/) (2022)
- [TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Prediction](https://mlsanity.com/paper/2307.16106/) (2023)
- [A Review of Prospects and Opportunities in Disassembly with Human-Robot Collaboration](https://mlsanity.com/paper/2310.13643/) (2023)
- [DE-TGN: Uncertainty-Aware Human Motion Forecasting using Deep Ensembles](https://mlsanity.com/paper/2307.03610/) (2023)
- [Integrating Uncertainty-Aware Human Motion Prediction into Graph-Based Manipulator Motion Planning](https://mlsanity.com/paper/2405.09779/) (2024)

## Built on by (most cited first, 2 of 2)

- [Warm-Starting Optimization-Based Motion Planning for Robotic Manipulators via Point Cloud-Conditioned Flow Matching](https://mlsanity.com/paper/2510.03460/) (2025)
- [KHMP: Frequency-Domain Kalman Refinement for High-Fidelity Human Motion Prediction](https://mlsanity.com/paper/2603.21327/) (2026)

## Abstract (arXiv)

Human motion prediction is a cornerstone of human-robot collaboration (HRC), as robots need to infer the future movements of human workers based on past motion cues to proactively plan their motion, ensuring safety in close collaboration scenarios. The diffusion model has demonstrated remarkable performance in predicting high-quality motion samples with reasonable diversity, but suffers from a slow generative process which necessitates multiple model evaluations, hindering real-world applications. To enable real-time prediction, in this work, we propose training a one-step multi-layer perceptron-based (MLP-based) diffusion model for motion prediction using knowledge distillation and Bayesian optimization. Our method contains two steps. First, we distill a pretrained diffusion-based motion predictor, TransFusion, directly into a one-step diffusion model with the same denoiser architecture. Then, to further reduce the inference time, we remove the computationally expensive components from the original denoiser and use knowledge distillation once again to distill the obtained one-step diffusion model into an even smaller model based solely on MLPs. Bayesian optimization is used to tune the hyperparameters for training the smaller diffusion model. Extensive experimental studies are conducted on benchmark datasets, and our model can significantly improve the inference speed, achieving real-time prediction without noticeable degradation in performance.