# OmniFlow: Human Omnidirectional Optical Flow

arXiv: https://arxiv.org/abs/2104.07960 · HTML: https://mlsanity.com/paper/2104.07960/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 16 Apr 2021.

## Builds on (most cited first, 10 of 10)

- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [Learning Multi-Human Optical Flow](https://mlsanity.com/paper/1910.11667/) (2019)
- [Learning from THEODORE: A Synthetic Omnidirectional Top-View Indoor Dataset for Deep Transfer Learning](https://mlsanity.com/paper/2011.05719/) (2020)
- [Optical Flow Dataset and Benchmark for Visual Crowd Analysis](https://mlsanity.com/paper/1811.07170/) (2018)
- [Learning Human Optical Flow](https://mlsanity.com/paper/1806.05666/) (2018)

## Built on by (most cited first, 9 of 9)

- [Review on Panoramic Imaging and Its Applications in Scene Understanding](https://mlsanity.com/paper/2205.05570/) (2022)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [PanoFlow: Learning 360deg Optical Flow for Surrounding Temporal Understanding](https://mlsanity.com/paper/2202.13388/) (2022)
- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [Human Pose Estimation in Monocular Omnidirectional Top-View Images](https://mlsanity.com/paper/2304.08186/) (2023)
- [NToP: NeRF-Powered Large-scale Dataset Generation for 2D and 3D Human Pose Estimation in Top-View Fisheye Images](https://mlsanity.com/paper/2402.18196/) (2024)
- [Learning Omnidirectional Flow in 360-degree Video via Siamese Representation](https://mlsanity.com/paper/2208.03620/) (2022)
- [Optical Flow Estimation in 360^circ Videos: Dataset, Model and Application](https://mlsanity.com/paper/2301.11880/) (2023)
- [Cell as Point: One-Stage Framework for Efficient Cell Tracking](https://mlsanity.com/paper/2411.14833/) (2024)

## Abstract (arXiv)

Optical flow is the motion of a pixel between at least two consecutive video frames and can be estimated through an end-to-end trainable convolutional neural network. To this end, large training datasets are required to improve the accuracy of optical flow estimation. Our paper presents OmniFlow: a new synthetic omnidirectional human optical flow dataset. Based on a rendering engine we create a naturalistic 3D indoor environment with textured rooms, characters, actions, objects, illumination and motion blur where all components of the environment are shuffled during the data capturing process. The simulation has as output rendered images of household activities and the corresponding forward and backward optical flow. To verify the data for training volumetric correspondence networks for optical flow estimation we train different subsets of the data and test on OmniFlow with and without Test-Time-Augmentation. As a result we have generated 23,653 image pairs and corresponding forward and backward optical flow. Our dataset can be downloaded from: https://mytuc.org/byfs