mlsanityData to 7 Oct 2026

BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks

arXiv 1709.01686 ↗First posted 6 Sep 2017cs.NE cs.CV cs.LGcited by 591

Not on the map. Why?

Cited by 120 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep neural networks are state of the art methods for many learning tasks due to their ability to extract increasingly better features at each network layer. However, the improved performance of additional layers in a deep network comes at the cost of added latency and energy usage in feedforward inference. As networks continue to get deeper and larger, these costs become more prohibitive for real-time and energy-sensitive applications. To address this issue, we present BranchyNet, a novel deep network architecture that is augmented with additional side branch classifiers. The architecture allows prediction results for a large portion of test samples to exit the network early via these branches when samples can already be inferred with high confidence. BranchyNet exploits the observation that features learned at an early layer of a network may often be sufficient for the classification of many data points. For more difficult samples, which are expected less frequently, BranchyNet will use further or all network layers to provide the best likelihood of correct prediction. We study the BranchyNet architecture using several well-known networks (LeNet, AlexNet, ResNet) and datasets (MNIST, CIFAR10) and show that it can both improve accuracy and significantly reduce the inference time of the network.

Key facts. Cited by 591 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 68; 2022: 76; 2023: 94; 2024: 99; 2025: 119; 2026: 135.

591 papers.

Sort:

By area:

  1. 2023LightGlue: Local Feature Matching at Light Speedcited by 583
  2. 2021Dynamic Neural Networks: A Surveycited by 405
  3. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  4. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 360
  5. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
  6. 2022Confident Adaptive Language Modelingcited by 297
  7. 2021Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognitioncited by 143
  8. 2021Deep Learning Through the Lens of Example Difficultycited by 139
  9. 2025Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computationcited by 120
  10. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decodingcited by 118
  11. 2024DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Executioncited by 116
  12. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challengescited by 114
38 more
  1. 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
  2. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 101
  3. 2024Not All Layers of LLMs Are Necessary During Inferencecited by 95
  4. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 87
  5. 2022Head2Toe: Utilizing Intermediate Representations for Better Transfer Learningcited by 83
  6. 2023Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingcited by 80
  7. 2025Universal Model Routing for Efficient LLM Inferencecited by 80
  8. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 78
  9. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 74
  10. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 73
  11. 2024Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Modelscited by 72
  12. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
  13. 2024Dynamic Diffusion Transformercited by 60
  14. 2021FrameExit: Conditional Early Exiting for Efficient Video Recognitioncited by 59
  15. 2023When Does Confidence-Based Cascade Deferral Suffice?cited by 58
  16. 2022EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruningcited by 56
  17. 2022Dynamic Multimodal Fusioncited by 54
  18. 2023A Dynamic Multi-Scale Voxel Flow Network for Video Predictioncited by 53
  19. 2021A Survey on Green Deep Learningcited by 52
  20. 2023Adaptive Rotated Convolution for Rotated Object Detectioncited by 51
  21. 2022A Survey on Model Compression and Acceleration for Pretrained Language Modelscited by 45
  22. 2022Learning to Weight Samples for Dynamic Early-exiting Networkscited by 44
  23. 2024A Survey on Transformer Compressioncited by 44
  24. 2022You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Modelcited by 42
  25. 2021Zero Time Waste: Recycling Predictions in Early Exit Neural Networkscited by 39
  26. 2023SEENN: Towards Temporal Spiking Early-Exit Neural Networkscited by 39
  27. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 38
  28. 2023Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorizationcited by 34
  29. 2022Path Independent Equilibrium Models Can Better Exploit Test-Time Computationcited by 33
  30. 2024Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanismcited by 31
  31. 2023LGViT: Dynamic Early Exiting for Accelerating Vision Transformercited by 30
  32. 2024M'elange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneitycited by 30
  33. 2021Auto-Split: A General Framework of Collaborative Edge-Cloud AIcited by 29
  34. 2025Learning Unmasking Policies for Diffusion Language Modelscited by 29
  35. 2021Multi-Exit Vision Transformer for Dynamic Inferencecited by 28
  36. 2021ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Trainingcited by 27
  37. 2024Fast yet Safe: Early-Exiting with Risk Controlcited by 27
  38. 2021Single-Layer Vision Transformers for More Accurate Early Exits with Less Overheadcited by 26

50 most cited of 591 All 591 (JSON)