mlsanityData to 14 Oct 2026

REWIND: Real-Time Egocentric Whole-Body Motion Diffusion with Exemplar-Based Identity Conditioning

arXiv 2504.04956 ↗First posted 7 Apr 2025cs.GR cs.CVcited by 14

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

We present REWIND (Real-Time Egocentric Whole-Body Motion Diffusion), a one-step diffusion model for real-time, high-fidelity human motion estimation from egocentric image inputs. While an existing method for egocentric whole-body (i.e., body and hands) motion estimation is non-real-time and acausal due to diffusion-based iterative motion refinement to capture correlations between body and hand poses, REWIND operates in a fully causal and real-time manner. To enable real-time inference, we introduce (1) cascaded body-hand denoising diffusion, which effectively models the correlation between egocentric body and hand motions in a fast, feed-forward manner, and (2) diffusion distillation, which enables high-quality motion estimation with a single denoising step. Our denoising diffusion model is based on a modified Transformer architecture, designed to causally model output motions while enhancing generalizability to unseen motion lengths. Additionally, REWIND optionally supports identity-conditioned motion estimation when identity prior is available. To this end, we propose a novel identity conditioning method based on a small set of pose exemplars of the target identity, which further enhances motion estimation quality. Through extensive experiments, we demonstrate that REWIND significantly outperforms the existing baselines both with and without exemplar-based identity conditioning.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 54 of its 65 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

54 of 65 references.

References · 54

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  9. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  10. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  11. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  12. 2022Human Motion Diffusion Modelcited by 1,080
38 more
  1. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  2. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  3. 2023Adversarial Diffusion Distillationcited by 759
  4. 2023Human Motion Diffusion as a Generative Priorcited by 357
  5. 2022EDGE: Editable Dance Generation From Musiccited by 287
  6. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  7. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 208
  8. 2021FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integrationcited by 169
  9. 2020Monocular Expressive Body Regression through Body-Driven Attentioncited by 155
  10. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 153
  11. 2021Collaborative Regression of Expressive Bodies using Moderationcited by 137
  12. 2021Real-time Deep Dynamic Characterscited by 133
  13. 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 114
  14. 2022AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensingcited by 110
  15. 2023Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Modelcited by 107
  16. 2023DiffCollage: Parallel Generation of Large Content with Diffusion Modelscited by 103
  17. 2022DiffPose: Toward More Reliable 3D Pose Estimationcited by 100
  18. 2022Ego-Body Pose Estimation via Ego-Head Pose Estimationcited by 94
  19. 2024Multi-Track Timeline Control for Text-Driven 3D Human Motion Generationcited by 91
  20. 2019You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactionscited by 82
  21. 2020Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimationcited by 81
  22. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 80
  23. 2022DiffPose: Multi-hypothesis Human Pose Estimation using Diffusion modelscited by 65
  24. 2023One-Step Diffusion Distillation via Deep Equilibrium Modelscited by 61
  25. 2024Estimating Body and Hand Motion in an Ego-sensed Worldcited by 58
  26. 2022UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capturecited by 57
  27. 2021Estimating Egocentric 3D Human Pose in Global Spacecited by 56
  28. 2020Monocular Real-time Full Body Capture with Inter-part Correlationscited by 53
  29. 2024Score-Guided Diffusion for 3D Human Recoverycited by 39
  30. 2023DiffPose: SpatioTemporal Diffusion Model for Video-Based Human Pose Estimationcited by 36
  31. 2020Body2Hands: Learning to Infer 3D Hands from Conversational Gesture Body Dynamicscited by 34
  32. 2023Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Viewscited by 33
  33. 2023Egocentric Whole-Body Motion Capture with FisheyeViT and Diffusion-Based Motion Refinementcited by 31
  34. 2024EgoGen: An Egocentric Synthetic Data Generatorcited by 27
  35. 2024HMD^2: Environment-aware Motion Generation from Single Egocentric Head-Mounted Devicecited by 27
  36. 20233D Human Pose Perception from Egocentric Stereo Videoscited by 23
  37. 2024HUMOS: Human Motion Model Conditioned on Body Shapecited by 20
  38. 2024Real-Time Simulated Avatar from Head-Mounted Sensorscited by 17

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 5.

14 papers.

Sort:

By area:

  1. 2025FlexSelect: Flexible Token Selection for Efficient Long Video Understandingcited by 18
  2. 2025Towards Cross-View Point Correspondence in Vision-Language Modelscited by 14
  3. 2025MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamicscited by 8
  4. 2025ECHO: Ego-Centric modeling of Human-Object interactionscited by 5
  5. 2026Egocentric Visibility-Aware Human Pose Estimationcited by 1
  6. 2026CIGPose: Causal Intervention Graph Neural Network for Whole-Body Pose Estimationcited by 1
  7. 2025LieHMR: Autoregressive Human Mesh Recovery with SO(3) Diffusioncited by 0
  8. 2025Cascaded Diffusion Framework for Probabilistic Coarse-to-Fine Hand Pose Estimationcited by 0
  9. 2025Mem-MLP: Real-Time 3D Human Motion Generation from Sparse Inputscited by 0
  10. 2025OmniEgoCap: Camera-Agnostic Sequence-Level Egocentric Motion Reconstructioncited by 0
  11. 2025Human Motion Estimation with Everyday Wearablescited by 0
  12. 2026EgoPoseVR: Spatiotemporal Multi-Modal Reasoning for Egocentric Full-Body Pose in Virtual Realitycited by 0
2 more
  1. 2026AG-EgoPose: Spatially Anchored Residual Correction with Action Context for Monocular Egocentric 3D Pose Estimationcited by 0
  2. 2026EgoRelight: Egocentric Human Capture and Illumination Recovery for Relightable and Photoreal Avatar Renderingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Estimating Ego-Body Pose from Doubly Sparse Egocentric Video Data
  2. 2026FUSION: Full-Body Unified Motion Prior for Body and Hands via Diffusion
  3. 2026ACE-Ego-Hand: Repurposing Video Diffusion Models for Occlusion-Robust Egocentric 3D Hand Motion Recovery
  4. 2023BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis
  5. 2023DiffusionPoser: Real-time Human Motion Reconstruction From Arbitrary Sparse Sensors Using Autoregressive Diffusion
  6. 2025Diffusion-based 3D Hand Motion Recovery with Intuitive Physics
  7. 2025Whole-Body Conditioned Egocentric Video Prediction
  8. 2024MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videos