# Self-Supervised Learning via Conditional Motion Propagation

arXiv: https://arxiv.org/abs/1903.11412 · HTML: https://mlsanity.com/paper/1903.11412/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2019.

## Built on by (most cited first, 20 of 20)

- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals](https://mlsanity.com/paper/2102.06191/) (2021)
- [Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization](https://mlsanity.com/paper/2205.07839/) (2022)
- [MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model](https://mlsanity.com/paper/2405.20222/) (2024)
- [Discovering Object Masks with Transformers for Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2206.06363/) (2022)
- [The Emergence of Objectness: Learning Zero-Shot Segmentation from Videos](https://mlsanity.com/paper/2111.06394/) (2021)
- [AnimateAnything: Consistent and Controllable Animation for Video Generation](https://mlsanity.com/paper/2411.10836/) (2024)
- [DisPose: Disentangling Pose Guidance for Controllable Human Image Animation](https://mlsanity.com/paper/2412.09349/) (2024)
- [InstantDrag: Improving Interactivity in Drag-based Image Editing](https://mlsanity.com/paper/2409.08857/) (2024)
- [Motion-Augmented Self-Training for Video Recognition at Smaller Scale](https://mlsanity.com/paper/2105.01646/) (2021)
- [Simulating Fluids in Real-World Still Images](https://mlsanity.com/paper/2204.11335/) (2022)
- [SurgSora: Object-Aware Diffusion Model for Controllable Surgical Video Generation](https://mlsanity.com/paper/2412.14018/) (2024)
- [Transcript to Video: Efficient Clip Sequencing from Texts](https://mlsanity.com/paper/2107.11851/) (2021)
- [MotionAgent: Fine-grained Controllable Video Generation via Motion Field Agent](https://mlsanity.com/paper/2502.03207/) (2025)
- [Image Editing with Diffusion Models: A Survey](https://mlsanity.com/paper/2504.13226/) (2025)
- [Optical Flow boosts Unsupervised Localization and Segmentation](https://mlsanity.com/paper/2307.13640/) (2023)
- [Self-supervised Learning of Echocardiographic Video Representations via Online Cluster Distillation](https://mlsanity.com/paper/2506.11777/) (2025)
- [Tokenizing Motion: A Generative Approach for Scene Dynamics Compression](https://mlsanity.com/paper/2410.09768/) (2024)
- [Point-PNG: Conditional Pseudo-Negatives Generation for Point Cloud Pre-Training](https://mlsanity.com/paper/2409.15832/) (2024)
- [BlazeBVD: Make Scale-Time Equalization Great Again for Blind Video Deflickering](https://mlsanity.com/paper/2403.06243/) (2024)

## Abstract (arXiv)

Intelligent agent naturally learns from motion. Various self-supervised algorithms have leveraged motion cues to learn effective visual representations. The hurdle here is that motion is both ambiguous and complex, rendering previous works either suffer from degraded learning efficacy, or resort to strong assumptions on object motions. In this work, we design a new learning-from-motion paradigm to bridge these gaps. Instead of explicitly modeling the motion probabilities, we design the pretext task as a conditional motion propagation problem. Given an input image and several sparse flow guidance vectors on it, our framework seeks to recover the full-image motion. Compared to other alternatives, our framework has several appealing properties: (1) Using sparse flow guidance during training resolves the inherent motion ambiguity, and thus easing feature learning. (2) Solving the pretext task of conditional motion propagation encourages the emergence of kinematically-sound representations that poss greater expressive power. Extensive experiments demonstrate that our framework learns structural and coherent features; and achieves state-of-the-art self-supervision performance on several downstream tasks including semantic segmentation, instance segmentation, and human parsing. Furthermore, our framework is successfully extended to several useful applications such as semi-automatic pixel-level annotation. Project page: "http://mmlab.ie.cuhk.edu.hk/projects/CMP/".