Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions
Not on the map. Why?
Abstract (arXiv)
We present CoopNet, an approach that improves the cooperation of co-trained networks by dynamically adapting the apportionment of gradient, to ensure equitable learning progress. It is applied to motion-aware self-supervised prediction of depth maps, by introducing a new hybrid loss, based on a distribution model of photo-metric reconstruction errors made by, on the one hand the depth + odometry paired networks, and on the other hand the optical flow network. This model essentially assumes that the pixels from moving objects (that must be discarded for training depth and odometry), correspond to those where the two reconstructions strongly disagree. We justify this model by theoretical considerations and experimental evidences. A comparative evaluation on KITTI and CityScapes datasets shows that CoopNet improves or is comparable to the state-of-the-art in depth, odometry and optical flow predictions.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 37 of its 45 references that are in the map. First posted on arXiv 8 May 2026.
Builds on
37 of 45 references.
References · 37
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2017Mask R-CNNcited by 7,259
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2015Spatial Transformer Networkscited by 1,423
- 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
- 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
- 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
- 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
- 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
- 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
25 more
- 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
- 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
- 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
- 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
- 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
- 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
- 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
- 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
- 2017Learning Depth from Monocular Videos using Direct Methodscited by 154
- 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
- 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
- 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
- 2020Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volumecited by 116
- 2016Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothnesscited by 101
- 2017Occlusion Aware Unsupervised Learning of Optical Flowcited by 101
- 2020Unsupervised Monocular Depth Learning in Dynamic Scenescited by 84
- 2020Self-Supervised Monocular Scene Flow Estimationcited by 64
- 2021Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistencycited by 58
- 2020Distilled Semantics for Comprehensive Scene Understanding from Videoscited by 39
- 2018Every Pixel Counts: Unsupervised Geometry Learning with Holistic 3D Motion Understandingcited by 24
- 2021Learning Depth via Leveraging Semantics: Self-supervised Monocular Depth Estimation with Both Implicit and Explicit Semantic Guidancecited by 19
- 2020Attentional Separation-and-Aggregation Network for Self-supervised Depth-Pose Learning in Dynamic Scenescited by 11
- 2021Self-supervised Depth Estimation Leveraging Global Perception and Geometric Smoothness Using On-board Videoscited by 7
- 2021Unsupervised Joint Learning of Depth, Optical Flow, Ego-motion from Videocited by 5
- 2021Revisiting Self-Supervised Monocular Depth Estimationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.