# Im2Flow: Motion Hallucination from Static Images for Action Recognition

arXiv: https://arxiv.org/abs/1712.04109 · HTML: https://mlsanity.com/paper/1712.04109/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2017.

## Built on by (most cited first, 24 of 24)

- [Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision](https://mlsanity.com/paper/2306.11719/) (2023)
- [Generative Image Dynamics](https://mlsanity.com/paper/2309.07906/) (2023)
- [Pixel Motion as Universal Representation for Robot Control](https://mlsanity.com/paper/2505.07817/) (2025)
- [Deep Unsupervised Key Frame Extraction for Efficient Video Classification](https://mlsanity.com/paper/2211.06742/) (2022)
- [Temporally smooth online action detection using cycle-consistent future anticipation](https://mlsanity.com/paper/2104.08030/) (2021)
- [VISAGE: Video Synthesis using Action Graphs for Surgery](https://mlsanity.com/paper/2410.17751/) (2024)
- [Dimensions of Motion: Monocular Prediction through Flow Subspaces](https://mlsanity.com/paper/2112.01502/) (2021)
- [Continuous Human Action Recognition for Human-Machine Interaction: A Review](https://mlsanity.com/paper/2202.13096/) (2022)
- [Objects do not disappear: Video object detection by single-frame object location anticipation](https://mlsanity.com/paper/2308.04770/) (2023)
- [Future Optical Flow Prediction Improves Robot Control & Video Generation](https://mlsanity.com/paper/2601.10781/) (2026)
- [Multi-Object Tracking with Hallucinated and Unlabeled Videos](https://mlsanity.com/paper/2108.08836/) (2021)
- [EDCFlow: Exploring Temporally Dense Difference Maps for Event-based Optical Flow Estimation](https://mlsanity.com/paper/2506.03512/) (2025)
- [Visual Semantic Parsing: From Images to Abstract Meaning Representation](https://mlsanity.com/paper/2210.14862/) (2022)
- [What If : Understanding Motion Through Sparse Interactions](https://mlsanity.com/paper/2510.12777/) (2025)
- [Envisioning the Future, One Step at a Time](https://mlsanity.com/paper/2604.09527/) (2026)
- [Efficient Human Vision Inspired Action Recognition using Adaptive Spatiotemporal Sampling](https://mlsanity.com/paper/2207.05249/) (2022)
- [DroneAttention: Sparse Weighted Temporal Attention for Drone-Camera Based Activity Recognition](https://mlsanity.com/paper/2212.03384/) (2022)
- [Unified Simulation, Perception, and Generation of Human Behavior](https://mlsanity.com/paper/2204.13678/) (2022)
- [Knowledge Guided Bidirectional Attention Network for Human-Object Interaction Detection](https://mlsanity.com/paper/2207.07979/) (2022)
- [SWTF: Sparse Weighted Temporal Fusion for Drone-Based Activity Recognition](https://mlsanity.com/paper/2211.05531/) (2022)
- [Enhancing Weakly-Supervised Object Detection on Static Images through (Hallucinated) Motion](https://mlsanity.com/paper/2409.09616/) (2024)
- [Group-DINOmics: Incorporating People Dynamics into DINO for Self-supervised Group Activity Feature Learning](https://mlsanity.com/paper/2604.04467/) (2026)
- [The JEPA Predictor: A Transferable Operator for Occluded Feature Completion](https://mlsanity.com/paper/2607.16274/) (2026)
- [Make It Flow: Animating Water in 3D Gaussian Splatting Scenes](https://mlsanity.com/paper/2610.13860/) (2026)

## Abstract (arXiv)

Existing methods to recognize actions in static images take the images at their face value, learning the appearances---objects, scenes, and body poses---that distinguish each action class. However, such models are deprived of the rich dynamic structure and motions that also define human activity. We propose an approach that hallucinates the unobserved future motion implied by a single snapshot to help static-image action recognition. The key idea is to learn a prior over short-term dynamics from thousands of unlabeled videos, infer the anticipated optical flow on novel static images, and then train discriminative models that exploit both streams of information. Our main contributions are twofold. First, we devise an encoder-decoder convolutional neural network and a novel optical flow encoding that can translate a static image into an accurate flow map. Second, we show the power of hallucinated flow for recognition, successfully transferring the learned motion into a standard two-stream network for activity recognition. On seven datasets, we demonstrate the power of the approach. It not only achieves state-of-the-art accuracy for dense optical flow prediction, but also consistently enhances recognition of actions and dynamic scenes.