mlsanityData to 7 Oct 2026

F-PABEE: Flexible-patience-based Early Exiting for Single-label and Multi-label text Classification Tasks

arXiv 2305.11916 ↗First posted 21 May 2023cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

Computational complexity and overthinking problems have become the bottlenecks for pre-training language models (PLMs) with millions or even trillions of parameters. A Flexible-Patience-Based Early Exiting method (F-PABEE) has been proposed to alleviate the problems mentioned above for single-label classification (SLC) and multi-label classification (MLC) tasks. F-PABEE makes predictions at the classifier and will exit early if predicted distributions of cross-layer are consecutively similar. It is more flexible than the previous state-of-the-art (SOTA) early exiting method PABEE because it can simultaneously adjust the similarity score thresholds and the patience parameters. Extensive experiments show that: (1) F-PABEE makes a better speedup-accuracy balance than existing early exiting strategies on both SLC and MLC tasks. (2) F-PABEE achieves faster inference and better performances on different PLMs such as BERT and ALBERT. (3) F-PABEE-JSKD performs best for F-PABEE with different similarity measures.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 27 of its 44 references that are in the map. First posted on arXiv 21 May 2023.

Builds on

27 of 44 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2019Are Sixteen Heads Really Better than One?cited by 803
  9. 2018Universal Transformerscited by 613
  10. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  11. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 502
  12. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 434
15 more
  1. 2021A Survey of Transformerscited by 399
  2. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
  3. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
  4. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 267
  5. 2021I-BERT: Integer-only BERT Quantizationcited by 250
  6. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 224
  7. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 175
  8. 2020The Right Tool for the Job: Matching Model and Instance Complexitiescited by 125
  9. 2020BERT-of-Theseus: Compressing BERT by Progressive Module Replacingcited by 112
  10. 2018SGM: Sequence Generation Model for Multi-label Classificationcited by 89
  11. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 73
  12. 2022A Survey on Dynamic Neural Networks for Natural Language Processingcited by 24
  13. 2022Continuous Detection, Rapidly React: Unseen Rumors Detection based on Continual Prompt-Tuningcited by 12
  14. 2018Shallow-Deep Networks: Understanding and Mitigating Network Overthinkingcited by 6
  15. 2020MVP-BERT: Redesigning Vocabularies for Chinese BERT and Multi-Vocab Pretrainingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 6; 2026: 6.

18 papers.

Sort:

By area:

  1. 2023PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domaincited by 40
  2. 2024ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Modelscited by 36
  3. 2024MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuningcited by 19
  4. 2025AMAS: Adaptively Determining Communication Topology for LLM-based Multi-Agent Systemcited by 15
  5. 2025PARA: Parameter-Efficient Fine-tuning with Prompt Aware Representation Adjustmentcited by 9
  6. 2025FT-MDT: Extracting Decision Trees from Medical Texts via a Novel Low-rank Adaptation Methodcited by 7
  7. 2026ADEPT: Adaptive Dynamic Early-Exit Process for Transformerscited by 4
  8. 2026MRAG: Benchmarking Retrieval-Augmented Generation for Bio-medicinecited by 4
  9. 2025Improving Prediction Certainty Estimation for Reliable Early Exiting via Null Space Projectioncited by 2
  10. 2023Improving Prompt Tuning with Learned Prompting Layerscited by 1
  11. 2024IAPT: Instruction-Aware Prompt Tuning for Large Language Modelscited by 1
  12. 2024COSEE: Consistency-Oriented Signal-Based Early Exiting via Calibrated Sample Weighting Mechanismcited by 1
6 more
  1. 2025EPEE: Towards Efficient and Effective Foundation Models in Biomedicinecited by 1
  2. 2025DeeAD: Dynamic Early Exit of Vision-Language Action for Efficient Autonomous Drivingcited by 1
  3. 2026Evaluating ChatGPT on Medical Information Extraction Tasks: Performance, Explainability and Beyondcited by 1
  4. 2026Pursuing Best Industrial Practices for Retrieval-Augmented Generation in the Medical Domaincited by 1
  5. 2026ACL: Aligned Contrastive Learning Improves BERT and Multi-exit BERT Fine-tuningcited by 1
  6. 2026A Generative Model for Joint Multiple Intent Detection and Slot Fillingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Early Exiting with Ensemble Internal Classifiers
  2. 2026Two-dimensional early exit optimisation of LLM inference
  3. 2026Discrete Diffusion Language Models Are Training-Free Multi-Label Classifiers
  4. 2025BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Experts
  5. 2022Fast and Accurate FSA System Using ELBERT: An Efficient and Lightweight BERT
  6. 2022Avoid Overthinking in Self-Supervised Models for Speech Recognition
  7. 2025Diffusion Disambiguation Models for Partial Label Learning
  8. 2022Rethinking Label Smoothing on Multi-hop Question Answering