mlsanityData to 7 Oct 2026

Deeply-Supervised Nets

arXiv 1409.5185 ↗First posted 18 Sep 2014stat.ML cs.CV cs.LGcited by 364

Not on the map. Why?

Abstract (arXiv)

Our proposed deeply-supervised nets (DSN) method simultaneously minimizes classification error while making the learning process of hidden layers direct and transparent. We make an attempt to boost the classification performance by studying a new formulation in deep networks. Three aspects in convolutional neural networks (CNN) style architectures are being looked at: (1) transparency of the intermediate layers to the overall classification; (2) discriminativeness and robustness of learned features, especially in the early layers; (3) effectiveness in training due to the presence of the exploding and vanishing gradients. We introduce "companion objective" to the individual hidden layers, in addition to the overall objective at the output layer (a different strategy to layer-wise pre-training). We extend techniques from stochastic gradient methods to analyze our algorithm. The advantage of our method is evident and our experimental result on benchmark datasets shows significant performance gain over existing methods (e.g. all state-of-the-art results on MNIST, CIFAR-10, CIFAR-100, and SVHN).

Key facts. Cited by 364 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Sep 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 79; 2022: 79; 2023: 61; 2024: 41; 2025: 48; 2026: 56.

364 papers.

Sort:

By area:

  1. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  2. 2024YOLOv9: Learning What You Want to Learn Using Programmable Gradient Informationcited by 422
  3. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  4. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 186
  5. 2024Swin-UMamba: Mamba-based UNet with ImageNet-based pretrainingcited by 179
  6. 2022Highly Accurate Dichotomous Image Segmentationcited by 110
  7. 2021Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsitycited by 98
  8. 2021MOOD: Multi-level Out-of-distribution Detectioncited by 94
  9. 2022PromptCAL: Contrastive Affinity Learning via Auxiliary Prompts for Generalized Novel Category Discoverycited by 85
  10. 2022Robustness to Unbounded Smoothness of Generalized SignSGDcited by 80
  11. 2021Line Segment Detection Using Transformers without Edgescited by 73
  12. 2021Revisiting Locally Supervised Learning: an Alternative to End-to-end Trainingcited by 71
38 more
  1. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
  2. 2022MPCFormer: fast, performant and private Transformer inference with MPCcited by 67
  3. 2021MT3: Meta Test-Time Training for Self-Supervised Test-Time Adaptioncited by 65
  4. 2021FrameExit: Conditional Early Exiting for Efficient Video Recognitioncited by 59
  5. 2023M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalitiescited by 56
  6. 2021MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentationcited by 53
  7. 2025End-to-End Test-Time Training for Long Contextcited by 53
  8. 2021GANav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environmentscited by 51
  9. 2021Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformerscited by 49
  10. 2023Advances in Deep Concealed Scene Understandingcited by 47
  11. 2022Understanding AdamW through Proximal Methods and Scale-Freenesscited by 46
  12. 2021Boundary-Aware Segmentation Network for Mobile and Web Applicationscited by 41
  13. 2022Local-Global Context Aware Transformer for Language-Guided Video Segmentationcited by 41
  14. 2024DRCT: Saving Image Super-resolution away from Information Bottleneckcited by 40
  15. 2021Every Annotation Counts: Multi-label Deep Supervision for Medical Image Segmentationcited by 37
  16. 2022Visual Attention Methods in Deep Learning: An In-Depth Surveycited by 37
  17. 2021Video-based Person Re-identification with Spatial and Temporal Memory Networkscited by 35
  18. 2022MTTrans: Cross-Domain Object Detection with Mean-Teacher Transformercited by 35
  19. 2021Shallow Feature Matters for Weakly Supervised Object Localizationcited by 34
  20. 2022Towards Data-Efficient Detection Transformerscited by 34
  21. 2023Dynamic Perceiver for Efficient Visual Recognitioncited by 33
  22. 2021TRACER: Extreme Attention Guided Salient Object Tracing Networkcited by 32
  23. 2022Reversible Column Networkscited by 32
  24. 2023CLUSTSEG: Clustering for Universal Segmentationcited by 32
  25. 2021Physics-Informed Neural Nets for Control of Dynamical Systemscited by 31
  26. 2022LE-UDA: Label-efficient unsupervised domain adaptation for medical image segmentationcited by 27
  27. 2021TransForensics: Image Forgery Localization with Dense Self-Attentioncited by 26
  28. 2021Sliced Recursive Transformercited by 26
  29. 2022Transformer based multiple instance learning for weakly supervised histopathology image segmentationcited by 26
  30. 2022Improving anatomical plausibility in medical image segmentation via hybrid graph neural networks: applications to chest x-ray analysiscited by 25
  31. 2023On the Implicit Bias of Adamcited by 25
  32. 2022Video Demoireing with Relation-Based Temporal Consistencycited by 24
  33. 2026Improved Baselines with Representation Autoencoderscited by 24
  34. 2021SNE-RoadSeg+: Rethinking Depth-Normal Translation and Deep Supervision for Freespace Detectioncited by 23
  35. 2022No Reason for No Supervision: Improved Generalization in Supervised Modelscited by 23
  36. 2021Focus U-Net: A novel dual attention-gated CNN for polyp segmentation during colonoscopycited by 20
  37. 2022Contrastive Deep Supervisioncited by 20
  38. 2023Get3DHuman: Lifting StyleGAN-Human into a 3D Generative Model using Pixel-aligned Reconstruction Priorscited by 19

50 most cited of 364 All 364 (JSON)