mlsanityData to 7 Oct 2026

Distilled Semantics for Comprehensive Scene Understanding from Videos

arXiv 2003.14030 ↗First posted 31 Mar 2020cs.CV cs.LGcited by 39

Not on the map. Why?

Abstract (arXiv)

Whole understanding of the surroundings is paramount to autonomous systems. Recent works have shown that deep neural networks can learn geometry (depth) and motion (optical flow) from a monocular video without any explicit supervision from ground truth annotations, particularly hard to source for these two tasks. In this paper, we take an additional step toward holistic scene understanding with monocular cameras by learning depth and motion alongside with semantics, with supervision for the latter provided by a pre-trained network distilling proxy ground truth images. We address the three tasks jointly by a) a novel training protocol based on knowledge distillation and self-supervision and b) a compact network architecture which enables efficient scene understanding on both power hungry GPUs and low-power embedded platforms. We thoroughly assess the performance of our framework and show that it yields state-of-the-art results for monocular depth estimation, optical flow and motion segmentation.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 6; 2023: 10; 2024: 6; 2025: 4; 2026: 2.

39 papers.

Sort:

By area:

  1. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
  2. 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
  3. 2021Fine-grained Semantics-aware Representation Enhancement for Self-supervised Monocular Depth Estimationcited by 55
  4. 2024Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Failcited by 45
  5. 2021Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completioncited by 39
  6. 2023Learning Depth Estimation for Transparent and Mirror Surfacescited by 34
  7. 2023Booster: a Benchmark for Depth from Images of Specular and Transparent Surfacescited by 33
  8. 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditionscited by 29
  9. 2021Learning optical flow from still imagescited by 28
  10. 2022DenseMTL: Cross-task Attention Mechanism for Dense Multi-task Learningcited by 25
  11. 2023RM-Depth: Unsupervised Learning of Recurrent Monocular Depth in Dynamic Scenescited by 22
  12. 2023MACARONS: Mapping And Coverage Anticipation with RGB Online Self-Supervisioncited by 18
27 more
  1. 2023EGA-Depth: Efficient Guided Attention for Self-Supervised Multi-Camera Depth Estimationcited by 16
  2. 2023DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibriumcited by 16
  3. 2023GasMono: Geometry-Aided Self-Supervised Monocular Depth Estimation for Indoor Scenescited by 13
  4. 2021X-Distill: Improving Self-Supervised Monocular Depth via Cross-Task Distillationcited by 12
  5. 20213D Hierarchical Refinement and Augmentation for Unsupervised Learning of Depth and Pose from Monocular Videocited by 11
  6. 2024The Third Monocular Depth Estimation Challengecited by 10
  7. 2021Moving SLAM: Fully Unsupervised Deep Learning in Non-Rigid Scenescited by 8
  8. 2021CamLessMonoDepth: Monocular Depth Estimation with Unknown Camera Parameterscited by 8
  9. 2021THE Benchmark: Transferable Representation Learning for Monocular Height Estimationcited by 8
  10. 2023Self-supervised Hypergraphs for Learning Multiple World Interpretationscited by 8
  11. 2025The Fourth Monocular Depth Estimation Challengecited by 8
  12. 2022MonoDVPS: A Self-Supervised Monocular Depth Estimation Approach to Depth-aware Video Panoptic Segmentationcited by 7
  13. 2023Multi-Task Hypergraphs for Semi-supervised Learning using Earth Observationscited by 6
  14. 2024ROOT: VLM based System for Indoor Scene Understanding and Beyondcited by 6
  15. 2025ViLAM: Distilling Vision-Language Reasoning into Attention Maps for Social Robot Navigationcited by 5
  16. 2021Sensor-Guided Optical Flowcited by 4
  17. 2024GAM-Depth: Self-Supervised Indoor Depth Estimation Leveraging a Gradient-Aware Mask and Semantic Constraintscited by 3
  18. 2025NTIRE 2025 Challenge on HR Depth from Images of Specular and Transparent Surfacescited by 3
  19. 2022ROIFormer: Semantic-Aware Region of Interest Transformer for Efficient Self-Supervised Monocular Depth Estimationcited by 2
  20. 2023Learning Good Features to Transfer Across Tasks and Domainscited by 2
  21. 2021Video2Skill: Adapting Events in Demonstration Videos to Skills in an Environment using Cyclic MDP Homomorphismscited by 1
  22. 2022Hybrid Transformer Based Feature Fusion for Self-Supervised Monocular Depth Estimationcited by 1
  23. 2026Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictionscited by 1
  24. 2026ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Devicecited by 1
  25. 2022Neural Camera Modelscited by 0
  26. 2024Multiple Random Masking Autoencoder Ensembles for Robust Multimodal Semi-supervised Learningcited by 0
  27. 2025A Survey on Efficiency Optimization Techniques for DNN-based Video Analytics: Process Systems, Algorithms, and Applicationscited by 0