mlsanityData to 7 Oct 2026

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

arXiv 1502.01852 ↗First posted 6 Feb 2015cs.CV cs.AI cs.LGcited by 3,590

Not on the map. Why?

Cited by 44 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on our PReLU networks (PReLU-nets), we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66%). To our knowledge, our result is the first to surpass human-level performance (5.1%, Russakovsky et al.) on this visual recognition challenge.

Key facts. Cited by 3,590 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Feb 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 789; 2022: 702; 2023: 663; 2024: 584; 2025: 521; 2026: 330.

3,590 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021Multiscale Vision Transformerscited by 896
  3. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  4. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  5. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  6. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  7. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  8. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  9. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  10. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  11. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  12. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
38 more
  1. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  2. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  3. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  4. 2022MONAI: An open-source framework for deep learning in healthcarecited by 438
  5. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 396
  6. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  7. 2021CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flowscited by 320
  8. 2023VeRA: Vector-based Random Matrix Adaptationcited by 294
  9. 2021Dataset Condensation with Differentiable Siamese Augmentationcited by 290
  10. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  11. 2024PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Modelscited by 261
  12. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 254
  13. 2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 251
  14. 2023From Sparse to Soft Mixtures of Expertscited by 245
  15. 2021WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognitioncited by 221
  16. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  17. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  18. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  19. 2021Projected GANs Converge Fastercited by 204
  20. 2021R-Drop: Regularized Dropout for Neural Networkscited by 202
  21. 2023ReLoRA: High-Rank Training Through Low-Rank Updatescited by 200
  22. 2021Accounting for Variance in Machine Learning Benchmarkscited by 195
  23. 2021Large-Scale Representation Learning on Graphs via Bootstrappingcited by 184
  24. 2022Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limitcited by 184
  25. 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
  26. 2025Training Deep Learning Models with Norm-Constrained LMOscited by 177
  27. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  28. 2023Understanding plasticity in neural networkscited by 170
  29. 2021LogME: Practical Assessment of Pre-trained Models for Transfer Learningcited by 168
  30. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
  31. 2022Compute Trends Across Three Eras of Machine Learningcited by 161
  32. 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 156
  33. 2021NeRF--: Neural Radiance Fields Without Known Camera Parameterscited by 155
  34. 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 155
  35. 2021Adversarial Reciprocal Points Learning for Open Set Recognitioncited by 153
  36. 2021Activation Functions in Deep Learning: A Comprehensive Survey and Benchmarkcited by 153
  37. 2023A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAScited by 149
  38. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 147

50 most cited of 3,590 All 3,590 (JSON)