mlsanityData to 7 Oct 2026

SemARFlow: Injecting Semantics into Unsupervised Optical Flow Estimation for Autonomous Driving

arXiv 2303.06209 ↗First posted 10 Mar 2023cs.CV cs.ROcited by 9

Not on the map. Why?

Abstract (arXiv)

Unsupervised optical flow estimation is especially hard near occlusions and motion boundaries and in low-texture regions. We show that additional information such as semantics and domain knowledge can help better constrain this problem. We introduce SemARFlow, an unsupervised optical flow network designed for autonomous driving data that takes estimated semantic segmentation masks as additional inputs. This additional information is injected into the encoder and into a learned upsampler that refines the flow output. In addition, a simple yet effective semantic augmentation module provides self-supervision when learning flow and its boundaries for vehicles, poles, and sky. Together, these injections of semantic information improve the KITTI-2015 optical flow test error rate from 11.80% to 8.38%. We also show visible improvements around object boundaries as well as a greater ability to generalize across datasets. Code is available at https://github.com/duke-vision/semantic-unsup-flow-release.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 63 of its 90 references that are in the map. First posted on arXiv 10 Mar 2023.

Builds on

63 of 90 references.

References · 63

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  9. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  10. 2016Pyramid Scene Parsing Networkcited by 2,472
  11. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  12. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
38 more
  1. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  2. 2021Transformers in Vision: A Surveycited by 1,009
  3. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  4. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  5. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  6. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  7. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  8. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  9. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 421
  10. 2020A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospectscited by 396
  11. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 373
  12. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  13. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 324
  14. 2020BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentationcited by 294
  15. 2018Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomizationcited by 292
  16. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 223
  17. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  18. 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
  19. 2020Hierarchical Multi-Scale Attention for Semantic Segmentationcited by 183
  20. 2017Playing for Benchmarkscited by 169
  21. 2017Understanding Convolution for Semantic Segmentationcited by 168
  22. 2018Improving Semantic Segmentation via Video Propagation and Label Relaxationcited by 134
  23. 2019Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimationcited by 131
  24. 2019SelFlow: Self-Supervised Learning of Optical Flowcited by 128
  25. 2020What Matters in Unsupervised Optical Flowcited by 124
  26. 2020Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimationcited by 120
  27. 2020Flow-edge Guided Video Completioncited by 104
  28. 2016Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothnesscited by 101
  29. 2017Occlusion Aware Unsupervised Learning of Optical Flowcited by 101
  30. 2018Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimationcited by 94
  31. 2018SegStereo: Exploiting Semantic Information for Disparity Estimationcited by 88
  32. 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flowcited by 82
  33. 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
  34. 2021InverseForm: A Loss Function for Structured Boundary-Aware Segmentationcited by 65
  35. 2019DDFlow: Learning Optical Flow with Unlabeled Data Distillationcited by 63
  36. 2021High-Resolution Optical Flow from 1D Attention and Correlationcited by 54
  37. 2018SDCNet: Video Prediction Using Spatially-Displaced Convolutioncited by 50
  38. 2020UPFlow: Upsampling Pyramid for Unsupervised Optical Flow Learningcited by 49

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 2; 2026: 3.

9 papers.

  1. 2025TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motioncited by 9
  2. 2024UnSAMFlow: Unsupervised Optical Flow Guided by Segment Anything Modelcited by 8
  3. 2024Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptationcited by 4
  4. 2025PriOr-Flow: Enhancing Primitive Panoramic Optical Flow with Orthogonal Viewcited by 4
  5. 2024DREAM: Improving Video-Text Retrieval Through Relevance-Based Augmentation Using Large Foundation Modelscited by 3
  6. 2023UFD-PRiME: Unsupervised Joint Learning of Optical Flow and Stereo Depth through Pixel-Level Rigid Motion Estimationcited by 1
  7. 2026Two-Stream Interactive Joint Learning of Scene Parsing and Geometric Vision Taskscited by 0
  8. 2026GenOpticalFlow: A Generative Approach to Unsupervised Optical Flow Learningcited by 0
  9. 2026U2Flow: Uncertainty-Aware Unsupervised Optical Flow Estimationcited by 0