# H-MoRe: Learning Human-centric Motion Representation for Action Analysis

arXiv: https://arxiv.org/abs/2504.10676 · HTML: https://mlsanity.com/paper/2504.10676/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 80 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Builds on (most cited first, 30 of 54)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors](https://mlsanity.com/paper/2310.12190/) (2023)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation](https://mlsanity.com/paper/2306.07954/) (2023)
- [PointNetLK: Robust & Efficient Point Cloud Registration using PointNet](https://mlsanity.com/paper/1903.05711/) (2019)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time](https://mlsanity.com/paper/2211.03375/) (2022)
- [MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance](https://mlsanity.com/paper/2406.19680/) (2024)
- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [RPM-Net: Robust Point Matching using Learned Features](https://mlsanity.com/paper/2003.13479/) (2020)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose](https://mlsanity.com/paper/2303.07399/) (2023)
- [Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation](https://mlsanity.com/paper/2011.01461/) (2020)
- [GaitGraph: Graph Convolutional Network for Skeleton-Based Gait Recognition](https://mlsanity.com/paper/2101.11228/) (2021)
- [FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation](https://mlsanity.com/paper/2303.01237/) (2023)
- [Evidential Deep Learning for Open Set Action Recognition](https://mlsanity.com/paper/2107.10161/) (2021)
- [OpenGait: Revisiting Gait Recognition Toward Better Practicality](https://mlsanity.com/paper/2211.06597/) (2022)

## Built on by (most cited first, 8 of 8)

- [BiggerGait: Unlocking Gait Recognition with Layer-wise Representations from Large Vision Models](https://mlsanity.com/paper/2505.18132/) (2025)
- [LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing](https://mlsanity.com/paper/2507.00029/) (2025)
- [Procedural Mistake Detection via Action Effect Modeling](https://mlsanity.com/paper/2512.03474/) (2025)
- [Silhouette-based Gait Foundation Model](https://mlsanity.com/paper/2512.00691/) (2025)
- [ViTs for Action Classification in Videos: An Approach to Risky Tackle Detection in American Football Practice Videos](https://mlsanity.com/paper/2604.01318/) (2026)
- [Context-Aware Network Based on Multi-scale Spatio-temporal Attention for Action Recognition in Videos](https://mlsanity.com/paper/2512.18750/) (2025)
- [H-Flow: Self-supervised Human Scene Flow via Physics-inspired Joint Multi-modal Learning](https://mlsanity.com/paper/2605.22629/) (2026)
- [Pose-Anchored Optical Flow for Low-Latency Human Action Anticipation in Human-Robot Teaming](https://mlsanity.com/paper/2608.25495/) (2026)

## Abstract (arXiv)

In this paper, we propose H-MoRe, a novel pipeline for learning precise human-centric motion representation. Our approach dynamically preserves relevant human motion while filtering out background movement. Notably, unlike previous methods relying on fully supervised learning from synthetic data, H-MoRe learns directly from real-world scenarios in a self-supervised manner, incorporating both human pose and body shape information. Inspired by kinematics, H-MoRe represents absolute and relative movements of each body point in a matrix format that captures nuanced motion details, termed world-local flows. H-MoRe offers refined insights into human motion, which can be integrated seamlessly into various action-related applications. Experimental results demonstrate that H-MoRe brings substantial improvements across various downstream tasks, including gait recognition(CL@R1: +16.01%), action recognition(Acc@1: +8.92%), and video generation(FVD: -67.07%). Additionally, H-MoRe exhibits high inference efficiency (34 fps), making it suitable for most real-time scenarios. Models and code will be released upon publication.