SegFlow: Joint Learning for Video Object Segmentation and Optical Flow
Not on the map. Why?
Abstract (arXiv)
This paper proposes an end-to-end trainable network, SegFlow, for simultaneously predicting pixel-wise object segmentation and optical flow in videos. The proposed SegFlow has two branches where useful information of object segmentation and optical flow is propagated bidirectionally in a unified framework. The segmentation branch is based on a fully convolutional network, which has been proved effective in image segmentation task, and the optical flow branch takes advantage of the FlowNet model. The unified framework is trained iteratively offline to learn a generic notion, and fine-tuned online for specific objects. Extensive experiments on both the video object segmentation and optical flow datasets demonstrate that introducing optical flow improves the performance of segmentation and vice versa, against the state-of-the-art algorithms.
Key facts. Cited by 92 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Sep 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 28; 2022: 23; 2023: 24; 2024: 6; 2025: 9; 2026: 2.
92 papers.
Sort:
By area:
- 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
- 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
- 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
- 2022Towards An End-to-End Framework for Flow-Guided Video Inpaintingcited by 113
- 2021SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentationcited by 106
- 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
- 2021Full-Duplex Strategy for Video Object Segmentationcited by 78
- 2021STEP: Segmenting and Tracking Every Pixelcited by 73
- 2021Hierarchical Memory Matching Network for Video Object Segmentationcited by 72
- 2022Segmenting Moving Objects via an Object-Centric Layered Representationcited by 63
- 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 62
- 2022DIP: Deep Inverse Patchmatch for High-Resolution Optical Flowcited by 40
38 more
- 2022Per-Clip Video Object Segmentationcited by 40
- 2022Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentationcited by 37
- 2021Reliable Propagation-Correction Modulation for Video Object Segmentationcited by 36
- 2022Learning Quality-aware Dynamic Memory for Video Object Segmentationcited by 35
- 2022Look Before You Match: Instance Understanding Matters in Video Object Segmentationcited by 34
- 2022EM-driven unsupervised learning for efficient motion segmentationcited by 33
- 2021Self-supervised Multi-view Stereo via Effective Co-Segmentation and Data-Augmentationcited by 31
- 2021AuxAdapt: Stable and Efficient Test-Time Adaptation for Temporally Consistent Video Semantic Segmentationcited by 31
- 2022Unsupervised Video Object Segmentation via Prototype Memory Networkcited by 30
- 2021Iteratively Selecting an Easy Reference Frame Makes Unsupervised Video Object Segmentation Easiercited by 27
- 2022PVO: Panoptic Visual Odometrycited by 26
- 2022OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Cloudscited by 25
- 2022Vision+X: A Survey on Multimodal Learning in the Light of Datacited by 22
- 2022Global Spectral Filter Memory Network for Video Object Segmentationcited by 22
- 2021VIL-100: A New Dataset and A Baseline Model for Video Instance Lane Detectioncited by 21
- 2021HODOR: High-level Object Descriptors for Object Re-segmentation in Video Learned from Static Imagescited by 21
- 2023Guided Slot Attention for Unsupervised Video Object Segmentationcited by 19
- 2021Multi-Source Fusion and Automatic Predictor Selection for Zero-Shot Video Object Segmentationcited by 18
- 2022MulViMotion: Shape-aware 3D Myocardial Motion Tracking from Multi-View Cardiac MRIcited by 16
- 2023MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillationcited by 16
- 2022A Simple and Powerful Global Optimization for Unsupervised Video Object Segmentationcited by 15
- 2025Exploiting Temporal State Space Sharing for Video Semantic Segmentationcited by 15
- 2022BATMAN: Bilateral Attention Transformer in Motion-Appearance Neighboring Space for Video Object Segmentationcited by 14
- 2021Video Salient Object Detection via Contrastive Features and Attention Modulescited by 13
- 2022SiamMask: A Framework for Fast Online Object Tracking and Segmentationcited by 13
- 2021Video Instance Segmentation by Instance Flow Assemblycited by 12
- 2022Towards Robust Video Object Segmentation with Adaptive Object Calibrationcited by 11
- 2021Accelerating Video Object Segmentation with Compressed Videocited by 10
- 2022Implicit Motion-Compensated Network for Unsupervised Video Object Segmentationcited by 10
- 2021Learning to Better Segment Objects from Unseen Classes with Unlabeled Videoscited by 9
- 2023Co-attention Propagation Network for Zero-Shot Video Object Segmentationcited by 9
- 2023Tracking Anything in High Qualitycited by 9
- 2023CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentationcited by 9
- 2022The Right Spin: Learning Object Motion from Rotation-Compensated Flow Fieldscited by 8
- 2023Event-Free Moving Object Segmentation from Moving Ego Vehiclecited by 8
- 2023Amodal Optical Flowcited by 8
- 2024UnSAMFlow: Unsupervised Optical Flow Guided by Segment Anything Modelcited by 8
- 2021MUNet: Motion Uncertainty-aware Semi-supervised Video Object Segmentationcited by 7
50 most cited of 92 All 92 (JSON)
- 2026Temporal Aware Pruning for Efficient Diffusion-based Video Generation18 May 2026
- 2026RS-SSM: Refining Forgotten Specifics in State Space Model for Video Semantic Segmentation25 Mar 2026
- 2025A Study of Finetuning Video Transformers for Multi-view Geometry Tasks21 Dec 2025
- 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes7 Aug 2025
- 2025CMRINet: Joint Groupwise Registration and Segmentation for Cardiac Function Quantification from Cine-MRI22 May 2025
- 2025RGB-D Video Object Segmentation via Enhanced Multi-store Feature Memory23 Apr 2025
- 2025ZS-VCOS: Zero-Shot Video Camouflaged Object Segmentation By Optical Flow and Open Vocabulary Object Detection10 Apr 2025
- 2025Exploiting Temporal State Space Sharing for Video Semantic Segmentation26 Mar 2025
- 2025Investigation of Frame Differences as Motion Cues for Video Object Segmentation12 Mar 2025
- 2025Joint 3D Point Cloud Segmentation using Real-Sim Loop: From Panels to Trees and Branches7 Mar 2025
- 2025Learning Street View Representations with Spatiotemporal Contrast7 Feb 2025
- 2024Breaking The Ice: Video Segmentation for Close-Range Ice-Covered Waters7 Nov 2024
38 more
- 2024Configurable Embodied Data Generation for Class-Agnostic RGB-D Video Segmentation16 Oct 2024
- 2024Convolutional Networks as Extremely Small Foundation Models: Visual Prompting and Theoretical Perspective3 Sep 2024
- 2024UnSAMFlow: Unsupervised Optical Flow Guided by Segment Anything Model4 May 2024
- 2024Geometric Constraints in Deep Learning Frameworks: A Survey19 Mar 2024
- 2024ClickVOS: Click Video Object Segmentation10 Mar 2024
- 2023Amodal Optical Flow13 Nov 2023
- 2023Sketch-based Video Object Segmentation: Benchmark and Analysis13 Nov 2023
- 2023ISAR: A Benchmark for Single- and Few-Shot Object Instance Segmentation and Re-Identification5 Nov 2023
- 2023CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentation2 Nov 2023
- 2023SpVOS: Efficient Video Object Segmentation with Triple Sparse Convolution23 Oct 2023
- 2023Segmenting the motion components of a video: A long-term unsupervised model2 Oct 2023
- 2023Efficient Long-Short Temporal Attention Network for Unsupervised Video Object Segmentation21 Sep 2023
- 2023Multi-grained Temporal Prototype Learning for Few-shot Video Object Segmentation20 Sep 2023
- 2023StereoFlowGAN: Co-training for Stereo and Flow with Unsupervised Domain Adaptation4 Sep 2023
- 2023Joint Modeling of Feature, Correspondence, and a Compressed Memory for Video Object Segmentation25 Aug 2023
- 2023Learning Referring Video Object Segmentation from Weak Annotation4 Aug 2023
- 2023Tracking Anything in High Quality26 Jul 2023
- 2023Online Unsupervised Video Object Segmentation via Contrastive Motion Clustering21 Jun 2023
- 2023SplatFlow: Learning Multi-frame Optical Flow via Splatting15 Jun 2023
- 2023Event-Free Moving Object Segmentation from Moving Ego Vehicle28 Apr 2023
- 2023Co-attention Propagation Network for Zero-Shot Video Object Segmentation8 Apr 2023
- 2023Two-shot Video Object Segmentation21 Mar 2023
- 2023Adaptive Multi-source Predictor for Zero-shot Video Object Segmentation18 Mar 2023
- 2023Guided Slot Attention for Unsupervised Video Object Segmentation15 Mar 2023
- 2023InstMove: Instance Motion for Object-centric Video Segmentation14 Mar 2023
- 2023MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation14 Mar 2023
- 2023Camera-Radar Perception for Autonomous Vehicles and ADAS: Concepts, Datasets and Metrics8 Mar 2023
- 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenes3 Feb 2023
- 2023Maximal Cliques on Multi-Frame Proposal Graph for Unsupervised Video Object Segmentation29 Jan 2023
- 2022Learning a Fast 3D Spectral Approach to Object Segmentation and Tracking over Space and Time15 Dec 2022
- 2022Look Before You Match: Instance Understanding Matters in Video Object Segmentation13 Dec 2022
- 2022Global Spectral Filter Memory Network for Video Object Segmentation11 Oct 2022
- 2022OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Clouds10 Oct 2022
- 2022Vision+X: A Survey on Multimodal Learning in the Light of Data5 Oct 2022
- 2022A Simple and Powerful Global Optimization for Unsupervised Video Object Segmentation19 Sep 2022
- 2022Unsupervised Video Object Segmentation via Prototype Memory Network8 Sep 2022
- 2022How to Track and Segment Fish without Human Annotations: A Self-Supervised Deep Learning Approach23 Aug 2022
- 2022Per-Clip Video Object Segmentation3 Aug 2022
50 newest of 92 All 92 (JSON)
- 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
- 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
- 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
- 2022Towards An End-to-End Framework for Flow-Guided Video Inpaintingcited by 113
- 2021SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentationcited by 106
- 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
- 2021Full-Duplex Strategy for Video Object Segmentationcited by 78
- 2021STEP: Segmenting and Tracking Every Pixelcited by 73
- 2021Hierarchical Memory Matching Network for Video Object Segmentationcited by 72
- 2022Segmenting Moving Objects via an Object-Centric Layered Representationcited by 63
- 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 62
- 2022DIP: Deep Inverse Patchmatch for High-Resolution Optical Flowcited by 40
12 most cited of 89 All 89 (JSON)
- 2025A Study of Finetuning Video Transformers for Multi-view Geometry Tasks21 Dec 2025
- 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes7 Aug 2025
- 2025CMRINet: Joint Groupwise Registration and Segmentation for Cardiac Function Quantification from Cine-MRI22 May 2025
- 2025RGB-D Video Object Segmentation via Enhanced Multi-store Feature Memory23 Apr 2025
- 2025ZS-VCOS: Zero-Shot Video Camouflaged Object Segmentation By Optical Flow and Open Vocabulary Object Detection10 Apr 2025
- 2025Exploiting Temporal State Space Sharing for Video Semantic Segmentation26 Mar 2025
- 2025Investigation of Frame Differences as Motion Cues for Video Object Segmentation12 Mar 2025
- 2025Joint 3D Point Cloud Segmentation using Real-Sim Loop: From Panels to Trees and Branches7 Mar 2025
- 2025Learning Street View Representations with Spatiotemporal Contrast7 Feb 2025
- 2024Breaking The Ice: Video Segmentation for Close-Range Ice-Covered Waters7 Nov 2024
- 2024Configurable Embodied Data Generation for Class-Agnostic RGB-D Video Segmentation16 Oct 2024
- 2024Convolutional Networks as Extremely Small Foundation Models: Visual Prompting and Theoretical Perspective3 Sep 2024
12 newest of 89 All 89 (JSON)