mlsanityData to 7 Oct 2026

DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

arXiv 2004.12993 ↗First posted 27 Apr 2020cs.CL cs.LGcited by 322

Not on the map. Why?

Cited by 88 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large-scale pre-trained language models such as BERT have brought significant improvements to NLP applications. However, they are also notorious for being slow in inference, which makes them difficult to deploy in real-time applications. We propose a simple but effective method, DeeBERT, to accelerate BERT inference. Our approach allows samples to exit earlier without passing through the entire model. Experiments show that DeeBERT is able to save up to ~40% inference time with minimal degradation in model quality. Further analyses show different behaviors in the BERT transformer layers and also reveal their redundancy. Our work provides new ideas to efficiently apply deep transformer-based models to downstream tasks. Code is available at https://github.com/castorini/DeeBERT.

Key facts. Cited by 322 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 30; 2022: 47; 2023: 57; 2024: 61; 2025: 51; 2026: 76.

322 papers.

Sort:

By area:

  1. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  2. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  3. 2022Transformers in Time Series: A Surveycited by 489
  4. 2021Dynamic Neural Networks: A Surveycited by 405
  5. 2021A Survey of Transformerscited by 399
  6. 2022Confident Adaptive Language Modelingcited by 297
  7. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  8. 2022Structured Pruning Learns Compact and Accurate Modelscited by 167
  9. 2022A Fast Post-Training Pruning Framework for Transformerscited by 161
  10. 2021MoEfication: Transformer Feed-forward Layers are Mixtures of Expertscited by 149
  11. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  12. 2021Deep Learning Through the Lens of Example Difficultycited by 139
38 more
  1. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 125
  2. 2024DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Executioncited by 116
  3. 2022The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Modelscited by 115
  4. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challengescited by 114
  5. 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMscited by 111
  6. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  7. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 101
  8. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 94
  9. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 89
  10. 2023SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inferencecited by 89
  11. 2023Soft Merging of Experts with Adaptive Routingcited by 88
  12. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  13. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  14. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  15. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 78
  16. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 76
  17. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 74
  18. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 73
  19. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
  20. 2024Scaling Diffusion Transformers to 16 Billion Parameterscited by 65
  21. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 62
  22. 2023When Does Confidence-Based Cascade Deferral Suffice?cited by 58
  23. 2024MoE-I2: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decompositioncited by 57
  24. 2022EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruningcited by 56
  25. 2024Efficient Tuning and Inference for Large Language Models on Textual Graphscited by 54
  26. 2021A Survey on Green Deep Learningcited by 52
  27. 2021AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuningcited by 48
  28. 2022A Survey on Model Compression and Acceleration for Pretrained Language Modelscited by 45
  29. 2023LLMCad: Fast and Scalable On-device Large Language Model Inferencecited by 45
  30. 2021Towards Efficient NLP: A Standard Evaluation and A Strong Baselinecited by 44
  31. 2025AI Flow: Perspectives, Scenarios, and Approachescited by 44
  32. 2022Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systemscited by 43
  33. 2022You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Modelcited by 42
  34. 2023Towards the Law of Capacity Gap in Distilling Language Modelscited by 41
  35. 2021TR-BERT: Dynamic Token Reduction for Accelerating BERT Inferencecited by 40
  36. 2021BERTnesia: Investigating the capture and forgetting of knowledge in BERTcited by 40
  37. 2021Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compressioncited by 38
  38. 2021Towards Efficient Post-training Quantization of Pre-trained Language Modelscited by 37

50 most cited of 322 All 322 (JSON)