Data to 14 Oct 2026

FastBERT: a Self-distilling BERT with Adaptive Inference Time

arXiv 2004.02178 ↗First posted 5 Apr 2020cs.CLcited by 226

Not on the map. Why?

Cited by 63 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Pre-trained language models like BERT have proven to be highly performant. However, they are often computationally expensive in many practical scenarios, for such heavy models can hardly be readily implemented with limited resources. To improve their efficiency with an assured model performance, we propose a novel speed-tunable FastBERT with adaptive inference time. The speed at inference can be flexibly adjusted under varying demands, while redundant calculation of samples is avoided. Moreover, this model adopts a unique self-distillation mechanism at fine-tuning, further enabling a greater computational efficacy with minimal loss in performance. Our model achieves promising results in twelve English and Chinese datasets. It is able to speed up by a wide range from 1 to 12 times than BERT if given different speedup thresholds to make a speed-performance tradeoff.

Key facts. Cited by 226 papers in the map (data through 14 Oct 2026). First posted on arXiv 5 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 35; 2022: 42; 2023: 51; 2024: 41; 2025: 19; 2026: 38.

226 papers.

Sort:

By area:

2021Dynamic Neural Networks: A Surveycited by 408
2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 362
2022Confident Adaptive Language Modelingcited by 310
2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 224
2024A Survey on Efficient Inference for Large Language Modelscited by 205
2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
2022Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuningcited by 149
2021Deep Learning Through the Lens of Example Difficultycited by 141
2023AutoMix: Automatically Mixing Language Modelscited by 125
2024DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Executioncited by 117
2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challengescited by 115
2022The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Modelscited by 115

38 more

2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMscited by 115
2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 102
2024Not All Layers of LLMs Are Necessary During Inferencecited by 96
2023Soft Merging of Experts with Adaptive Routingcited by 90
2023SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inferencecited by 89
2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 81
2022Efficient Methods for Natural Language Processing: A Surveycited by 78
2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 75
2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 74
2021Time Series Prediction using Deep Learning Methods in Healthcarecited by 74
2024Harder Tasks Need More Experts: Dynamic Routing in MoE Modelscited by 72
2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 64
2021Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr"om Methodcited by 63
2022Learning Generalizable Models for Vehicle Routing Problems via Knowledge Distillationcited by 62
2023Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformerscited by 62
2023When Does Confidence-Based Cascade Deferral Suffice?cited by 60
2022EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruningcited by 56
2023PuMer: Pruning and Merging Tokens for Efficient Vision Language Modelscited by 53
2021A Survey on Green Deep Learningcited by 52
2021SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Imagescited by 50
2021AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuningcited by 48
2024A Survey on Transformer Compressioncited by 46
2022A Survey on Model Compression and Acceleration for Pretrained Language Modelscited by 45
2021Towards Efficient NLP: A Standard Evaluation and A Strong Baselinecited by 44
2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
2022You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Modelcited by 43
2021TR-BERT: Dynamic Token Reduction for Accelerating BERT Inferencecited by 40
2021HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformerscited by 39
2021Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compressioncited by 38
2022Path Independent Equilibrium Models Can Better Exploit Test-Time Computationcited by 38
2021NLP From Scratch Without Large-Scale Pretraining: A Simple and Efficient Frameworkcited by 34
2021Distilling Linguistic Context for Language Model Compressioncited by 33
2022A Simple Hash-Based Early Exiting Approach For Language Understanding and Generationcited by 32
2021Data-Free Knowledge Transfer: A Surveycited by 31
2023LGViT: Dynamic Early Exiting for Accelerating Vision Transformercited by 31
2021Accelerating BERT Inference for Sequence Labeling via Early-Exitcited by 29

50 most cited of 226 All 226 (JSON)