mlsanityData to 7 Oct 2026

Identity Mappings in Deep Residual Networks

arXiv 1603.05027 ↗First posted 16 Mar 2016cs.CV cs.LGcited by 1,960

Not on the map. Why?

Abstract (arXiv)

Deep residual networks have emerged as a family of extremely deep architectures showing compelling accuracy and nice convergence behaviors. In this paper, we analyze the propagation formulations behind the residual building blocks, which suggest that the forward and backward signals can be directly propagated from one block to any other block, when using identity mappings as the skip connections and after-addition activation. A series of ablation experiments support the importance of these identity mappings. This motivates us to propose a new residual unit, which makes training easier and improves generalization. We report improved results using a 1001-layer ResNet on CIFAR-10 (4.62% error) and CIFAR-100, and a 200-layer ResNet on ImageNet. Code is available at: https://github.com/KaimingHe/resnet-1k-layers

Key facts. Cited by 1,960 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Mar 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 1; 2018: 1; 2019: 1; 2020: 3; 2021: 543; 2022: 447; 2023: 364; 2024: 287; 2025: 199; 2026: 114.

1,960 papers.

Sort:

By area:

  1. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  2. 2022A ConvNet for the 2020scited by 3,545
  3. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  4. 2021Multiscale Vision Transformerscited by 896
  5. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  6. 2022A Generalist Agentcited by 802
  7. 2021Going deeper with Image Transformerscited by 759
  8. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  9. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  10. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  11. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  12. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
38 more
  1. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  2. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  3. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  4. 2021Implicit Behavioral Cloningcited by 442
  5. 2024YOLOv9: Learning What You Want to Learn Using Programmable Gradient Informationcited by 422
  6. 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 411
  7. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 396
  8. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  9. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
  10. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  11. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  12. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 334
  13. 2021Choose a Transformer: Fourier or Galerkincited by 333
  14. 2022How Do Vision Transformers Work?cited by 305
  15. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  16. 2021DeepViT: Towards Deeper Vision Transformercited by 288
  17. 2022YOLOv6: A Single-Stage Object Detection Framework for Industrial Applicationscited by 273
  18. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  19. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  20. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
  21. 2022CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimationcited by 237
  22. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  23. 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
  24. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  25. 2021Vision Transformers are Robust Learnerscited by 208
  26. 2021WildDeepfake: A Challenging Real-World Dataset for Deepfake Detectioncited by 207
  27. 2023PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solverscited by 205
  28. 2021Accounting for Variance in Machine Learning Benchmarkscited by 195
  29. 2022Towards Multi-spatiotemporal-scale Generalized PDE Modelingcited by 190
  30. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 177
  31. 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 173
  32. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  33. 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
  34. 2023A mathematical perspective on Transformerscited by 160
  35. 2021Revisiting ResNets: Improved Training and Scaling Strategiescited by 158
  36. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  37. 2022Benchmarking Self-Supervised Learning on Diverse Pathology Datasetscited by 148
  38. 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 148

50 most cited of 1,960 All 1,960 (JSON)