mlsanityData to 7 Oct 2026

A Comprehensive Review on Deep Supervision: Theories and Applications

arXiv 2207.02376 ↗First posted 6 Jul 2022cs.CV cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Deep supervision, or known as 'intermediate supervision' or 'auxiliary supervision', is to add supervision at hidden layers of a neural network. This technique has been increasingly applied in deep neural network learning systems for various computer vision applications recently. There is a consensus that deep supervision helps improve neural network performance by alleviating the gradient vanishing problem, as one of the many strengths of deep supervision. Besides, in different computer vision applications, deep supervision can be applied in different ways. How to make the most use of deep supervision to improve network performance in different applications has not been thoroughly investigated. In this paper, we provide a comprehensive in-depth review of deep supervision in both theories and applications. We propose a new classification of different deep supervision networks, and discuss advantages and limitations of current deep supervision networks in computer vision applications.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 38 of its 119 references that are in the map. First posted on arXiv 6 Jul 2022.

Builds on

38 of 119 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  8. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  9. 2019Object Detection in 20 Years: A Surveycited by 528
  10. 2014Deeply-Supervised Netscited by 364
  11. 2015Deeply-Recursive Convolutional Network for Image Super-Resolutioncited by 317
  12. 2020Component Divide-and-Conquer for Real-World Image Super-Resolutioncited by 243
26 more
  1. 2016Deeply supervised salient object detection with short connectionscited by 196
  2. 2018Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervisioncited by 167
  3. 2017Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networkscited by 138
  4. 2017Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentationcited by 109
  5. 20163D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumescited by 43
  6. 2017DSOD: Learning Deeply Supervised Object Detectors from Scratchcited by 41
  7. 2018Multi-Scale Structure-Aware Network for Human Pose Estimationcited by 37
  8. 2015Training Deeper Convolutional Networks with Deep Supervisioncited by 36
  9. 2021SNE-RoadSeg+: Rethinking Depth-Normal Translation and Deep Supervision for Freespace Detectioncited by 23
  10. 2018Object Detection from Scratch with Deep Supervisioncited by 21
  11. 2016Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsingcited by 19
  12. 2018Deep Supervision with Intermediate Conceptscited by 19
  13. 2020LabelEnc: A New Intermediate Supervision Method for Object Detectioncited by 18
  14. 2019DNA: Deeply-supervised Nonlinear Aggregation for Salient Object Detectioncited by 17
  15. 2018Semantic Edge Detection with Diverse Deep Supervisioncited by 13
  16. 2021HED-UNet: Combined Segmentation and Edge Detection for Monitoring the Antarctic Coastlinecited by 13
  17. 2018A Multi-stage Framework with Context Information Fusion Structure for Skin Lesion Segmentationcited by 12
  18. 2017Deep Supervision for Pancreatic Cyst Segmentation in Abdominal CT Scanscited by 10
  19. 2018Combating Uncertainty with Novel Losses for Automatic Left Atrium Segmentationcited by 7
  20. 2017Segmentation of Intracranial Arterial Calcification with Deeply Supervised Residual Dropout Networkscited by 5
  21. 2019Dilated deeply supervised networks for hippocampus segmentation in MRIcited by 5
  22. 2019Multi Scale Supervised 3D U-Net for Kidney and Tumor Segmentationcited by 4
  23. 2019Attention Deep Model with Multi-Scale Deep Supervision for Person Re-Identificationcited by 3
  24. 2021Parallel Multi-Scale Networks with Deep Supervision for Hand Keypoint Detectioncited by 2
  25. 2020MPG-Net: Multi-Prediction Guided Network for Segmentation of Retinal Layers in OCT Imagescited by 1
  26. 2021Multiview and Multiclass Image Segmentation using Deep Learning in Fetal Echocardiographycited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 0; 2025: 3; 2026: 4.

10 papers.

  1. 2023DeepMIM: Deep Supervision for Masked Image Modelingcited by 16
  2. 2023Solutions to Elliptic and Parabolic Problems via Finite Difference Based Unsupervised Small Linear Convolutional Neural Networkscited by 2
  3. 2026Verification of the Implicit World Model in a Generative Model via Adversarial Sequencescited by 1
  4. 2023Joint Learning for Scattered Point Cloud Understanding with Hierarchical Self-Distillationcited by 0
  5. 2025EU-Nets: Enhanced, Explainable and Parsimonious U-Netscited by 0
  6. 2025Multi-output Deep-Supervised Classifier Chains for Plant Pathologycited by 0
  7. 2025A Semantic Segmentation Algorithm for Pleural Effusion Based on DBIF-AUNetcited by 0
  8. 2026LION-DG: Layer-Informed Initialization with Deep Gradient Protocols for Accelerated Neural Network Trainingcited by 0
  9. 2026OmniISR: A Unified Framework for Centralized and Federated Learning via Intermediate Supervision and Regularizationcited by 0
  10. 2026From Gradient Clipping to Structural Refinement: Improving DPSGD for Medical Image Segmentationcited by 0