# Learning Human Optical Flow

arXiv: https://arxiv.org/abs/1806.05666 · HTML: https://mlsanity.com/paper/1806.05666/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2018.

## Built on by (most cited first, 7 of 7)

- [SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements](https://mlsanity.com/paper/2104.07660/) (2021)
- [HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences](https://mlsanity.com/paper/2103.15573/) (2021)
- [MDFlow: Unsupervised Optical Flow Learning by Reliable Mutual Knowledge Distillation](https://mlsanity.com/paper/2211.06018/) (2022)
- [OmniFlow: Human Omnidirectional Optical Flow](https://mlsanity.com/paper/2104.07960/) (2021)
- [Bootstrapping Human Optical Flow and Pose](https://mlsanity.com/paper/2210.15121/) (2022)
- [OPPH: A Vision-Based Operator for Measuring Body Movements for Personal Healthcare](https://mlsanity.com/paper/2408.09409/) (2024)
- [Using Motion Cues to Supervise Single-Frame Body Pose and Shape Estimation in Low Data Regimes](https://mlsanity.com/paper/2402.02736/) (2024)

## Abstract (arXiv)

The optical flow of humans is well known to be useful for the analysis of human action. Given this, we devise an optical flow algorithm specifically for human motion and show that it is superior to generic flow methods. Designing a method by hand is impractical, so we develop a new training database of image sequences with ground truth optical flow. For this we use a 3D model of the human body and motion capture data to synthesize realistic flow fields. We then train a convolutional neural network to estimate human flow fields from pairs of images. Since many applications in human motion analysis depend on speed, and we anticipate mobile applications, we base our method on SpyNet with several modifications. We demonstrate that our trained network is more accurate than a wide range of top methods on held-out test data and that it generalizes well to real image sequences. When combined with a person detector/tracker, the approach provides a full solution to the problem of 2D human flow estimation. Both the code and the dataset are available for research.