mlsanityData to 14 Oct 2026

Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motion

arXiv 2205.07844 ↗First posted 16 May 2022cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

Motion, measured via optical flow, provides a powerful cue to discover and learn objects in images and videos. However, compared to using appearance, it has some blind spots, such as the fact that objects become invisible if they do not move. In this work, we propose an approach that combines the strengths of motion-based and appearance-based segmentation. We propose to supervise an image segmentation network with the pretext task of predicting regions that are likely to contain simple motion patterns, and thus likely to correspond to objects. As the model only uses a single image as input, we can apply it in two settings: unsupervised video segmentation, and unsupervised image segmentation. We achieve state-of-the-art results for videos, and demonstrate the viability of our approach on still images containing novel objects. Additionally we experiment with different motion models and optical flow backbones and find the method to be robust to these change. Project page and code available at https://www.robots.ox.ac.uk/~vgg/research/gwm.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Builds on 66 of its 108 references that are in the map. First posted on arXiv 16 May 2022.

Builds on

66 of 108 references.

References · 66

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  6. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  7. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  8. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  9. 2017Deep Image Priorcited by 1,098
  10. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  11. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  12. 2020Object-Centric Learning with Slot Attentioncited by 876
38 more
  1. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  2. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  3. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 383
  4. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
  5. 2021Conditional Object-Centric Learning from Videocited by 248
  6. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  7. 2016One-Shot Video Object Segmentationcited by 235
  8. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  9. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  10. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 167
  11. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  12. 2021Illiterate DALL-E Learns to Composecited by 147
  13. 2020See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networkscited by 138
  14. 2019CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoningcited by 137
  15. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  16. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  17. 2021Self-Supervised Learning with Swin Transformerscited by 122
  18. 2020Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimationcited by 120
  19. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  20. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  21. 2021ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentationcited by 93
  22. 2017FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videoscited by 77
  23. 2019Unsupervised Object Segmentation by Redrawingcited by 74
  24. 2019Unsupervised Moving Object Detection via Contextual Information Separationcited by 72
  25. 2016It's Moving! A Probabilistic Model for Causal Motion Segmentation in Moving Camera Videoscited by 71
  26. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  27. 2020Betrayed by Motion: Camouflaged Object Discovery via Motion Segmentationcited by 62
  28. 2022Deformable Sprites for Unsupervised Video Decompositioncited by 61
  29. 2020Autoregressive Unsupervised Image Segmentationcited by 60
  30. 2017W-Net: A Deep Model for Fully Unsupervised Image Segmentationcited by 57
  31. 2019Emergence of Object Segmentation in Perturbed Generative Modelscited by 57
  32. 2020Generative Neurosymbolic Machinescited by 57
  33. 2022Unsupervised Salient Object Detection with Spectral Cluster Votingcited by 56
  34. 2020Object Segmentation Without Labels with Large-Scale Generative Modelscited by 52
  35. 2018Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspectivecited by 50
  36. 2020Learning Physical Graph Representations from Visual Scenescited by 50
  37. 2021Unsupervised Layered Image Decomposition into Object Prototypescited by 50
  38. 2021The Emergence of Objectness: Learning Zero-Shot Segmentation from Videoscited by 45

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 16; 2024: 9; 2025: 9; 2026: 5.

42 papers.

Sort:

By area:

  1. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
  2. 2023Diffusion Models for Open-Vocabulary Segmentationcited by 64
  3. 2024Moving Object Segmentation: All You Need Is SAM (and Flow)cited by 30
  4. 2023Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videoscited by 20
  5. 2022Unsupervised Multi-object Segmentation by Predicting Probable Motion Patternscited by 19
  6. 2023Object-centric Learning with Cyclic Walks between Parts and Wholecited by 16
  7. 2023Multi-Object Discovery by Low-Dimensional Object Motioncited by 16
  8. 2023Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Surveycited by 16
  9. 2023Bootstrapping Objectness from Videos by Relaxed Common Fate and Visual Groupingcited by 15
  10. 2025Learning segmentation from point trajectoriescited by 15
  11. 2022Motion-inductive Self-supervised Object Discovery in Videoscited by 14
  12. 2023The Making and Breaking of Camouflagecited by 14
30 more
  1. 2023Appearance-Based Refinement for Object-Centric Motion Segmentationcited by 11
  2. 2024PooDLe: Pooled and dense self-supervised learning from naturalistic videoscited by 11
  3. 2023Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentationcited by 10
  4. 2025Scene-Centric Unsupervised Panoptic Segmentationcited by 10
  5. 2023LOCATE: Self-supervised Object Discovery via Flow-guided Graph-cut and Bootstrapped Self-trainingcited by 8
  6. 2023Optical Flow boosts Unsupervised Localization and Segmentationcited by 7
  7. 2024On Moving Object Segmentation from Monocular Video with Transformerscited by 6
  8. 2024Unsupervised Object Discovery: A Comprehensive Survey and Unified Taxonomycited by 4
  9. 2025Slot-BERT: Self-supervised Object Discovery in Surgical Videocited by 4
  10. 2025Motion-Refined DINOSAUR for Unsupervised Multi-Object Discoverycited by 4
  11. 2023FODVid: Flow-guided Object Discovery in Videoscited by 3
  12. 2023MOFO: MOtion FOcused Self-Supervision for Video Understandingcited by 3
  13. 2023Segmenting the motion components of a video: A long-term unsupervised modelcited by 2
  14. 2023Mask as Supervision: Leveraging Unified Mask Information for Unsupervised 3D Pose Estimationcited by 2
  15. 2024CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformerscited by 2
  16. 2024MOD-UV: Learning Mobile Object Detectors from Unlabeled Videoscited by 2
  17. 2025FlowCut: Unsupervised Video Instance Segmentation via Temporal Mask Matchingcited by 2
  18. 2024Deep Spectral Improvement for Unsupervised Image Instance Segmentationcited by 1
  19. 2025AutoQ-VIS: Improving Unsupervised Video Instance Segmentation via Automatic Quality Assessmentcited by 1
  20. 2026GMOS: Grounding Moving Object Segmentation in 3D Space and Timecited by 1
  21. 2026Scene-Centric Unsupervised Video Panoptic Segmentationcited by 1
  22. 2022Improving Unsupervised Video Object Segmentation with Motion-Appearance Synergycited by 0
  23. 2024Dense Monocular Motion Segmentation Using Optical Flow and Pseudo Depth Map: A Zero-Shot Approachcited by 0
  24. 2024Improving Weakly-supervised Video Instance Segmentation by Leveraging Spatio-temporal Consistencycited by 0
  25. 2025Object Concepts Emerge from Motioncited by 0
  26. 2025Layered Motion Fusion: Lifting Motion Segmentation to 3D in Egocentric Videoscited by 0
  27. 2025Boosting Unsupervised Video Instance Segmentation with Automatic Quality-Guided Self-Trainingcited by 0
  28. 2026Latent Actions from Factorized Transition Effects under Agent Ambiguitycited by 0
  29. 2026WiFlow: Estimating Optical Flow using WiFi Channel State Informationcited by 0
  30. 2026Object Concepts Emerge from Motioncited by 0