mlsanityData to 14 Oct 2026

Dynamic Transformers Provide a False Sense of Efficiency

arXiv 2305.12228 ↗First posted 20 May 2023cs.CLcited by 14

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Despite much success in natural language processing (NLP), pre-trained language models typically lead to a high computational cost during inference. Multi-exit is a mainstream approach to address this issue by making a trade-off between efficiency and accuracy, where the saving of computation comes from an early exit. However, whether such saving from early-exiting is robust remains unknown. Motivated by this, we first show that directly adapting existing adversarial attack approaches targeting model accuracy cannot significantly reduce inference efficiency. To this end, we propose a simple yet effective attacking framework, SAME, a novel slowdown attack framework on multi-exit models, which is specially tailored to reduce the efficiency of the multi-exit models. By leveraging the multi-exit models' design characteristics, we utilize all internal predictions to guide the adversarial sample generation instead of merely considering the final prediction. Experiments on the GLUE benchmark show that SAME can effectively diminish the efficiency gain of various multi-exit models by 80% on average, convincingly validating its effectiveness and generalization ability.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 40 of its 55 references that are in the map. First posted on arXiv 20 May 2023.

Builds on

40 of 55 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Confident Adaptive Language Modeling14 Jul
  2. 2021Accelerating BERT Inference for Sequence Labeling via Early-Exit28 May

Other references · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  9. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  10. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  11. 2018Neural Network Acceptability Judgmentscited by 958
  12. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
26 more
  1. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  2. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  3. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
  4. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 463
  5. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 381
  6. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
  7. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
  8. 2019Q8BERT: Quantized 8Bit BERTcited by 300
  9. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  10. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 226
  11. 2016Crafting Adversarial Input Sequences for Recurrent Neural Networkscited by 103
  12. 2018Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examplescited by 94
  13. 2018On Adversarial Examples for Character-Level Neural Machine Translationcited by 85
  14. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  15. 2020A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inferencecited by 61
  16. 2022NICGSlowDown: Evaluating the Efficiency Robustness of Neural Image Caption Generation Modelscited by 50
  17. 2020Triple Wins: Boosting Accuracy, Robustness and Efficiency Together by Enabling Input-Adaptive Inferencecited by 45
  18. 2023Sibling-Attack: Rethinking Transferable Adversarial Attacks against Face Recognitioncited by 34
  19. 2022Character-level White-Box Adversarial Attacks against Transformers via Attachable Subwords Substitutioncited by 28
  20. 2022DeepPerform: An Efficient Approach for Performance Testing of Resource-Constrained Neural Networkscited by 17
  21. 2022LLMEffiChecker: Understanding and Testing Efficiency Degradation of Large Language Modelscited by 16
  22. 2022Perturbations in the Wild: Leveraging Human-Written Text Perturbations for Realistic Adversarial Attack and Defensecited by 13
  23. 2022Learning to Reverse DNNs from AI Programs Automaticallycited by 12
  24. 2022Genre-conditioned Acoustic Models for Automatic Lyrics Transcription of Polyphonic Musiccited by 7
  25. 2022PoLyScriber: Integrated Fine-tuning of Extractor and Lyrics Transcriber for Polyphonic Musiccited by 7
  26. 2022Predict-and-Update Network: Audio-Visual Speech Recognition Inspired by Human Speech Perceptioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 1; 2025: 6; 2026: 4.

14 papers.

Sort:

By area:

  1. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
  2. 2025CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Taskscited by 33
  3. 2025Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modificationscited by 21
  4. 2023RT-LM: Uncertainty-Aware Resource Management for Real-Time Inference of Language Modelscited by 11
  5. 2025LingoLoop Attack: Trapping MLLMs via Linguistic Context and State Entrapment into Endless Loopscited by 9
  6. 2023RED: A Systematic Real-Time Scheduling Approach for Robotic Environmental Dynamicscited by 6
  7. 2023R^3: On-device Real-Time Deep Reinforcement Learning for Autonomous Roboticscited by 5
  8. 2025Energy-Latency Attacks: A New Adversarial Threat to Deep Learningcited by 5
  9. 2025VLMInferSlow: Evaluating the Efficiency Robustness of Large Vision-Language Models as a Servicecited by 3
  10. 2025Efficiency Robustness of Dynamic Deep Learning Systemscited by 1
  11. 2026PIMbot: A Self-Adaptive Attack Framework for Adversarial Manipulation of Multi-Robot Reinforcement Learningcited by 1
  12. 2026MORE: Multi-Objective Adversarial Attacks on Speech Recognitioncited by 0
2 more
  1. 2026TetraRL: A Self-Adaptive Runtime for On-Device Deep Reinforcement Learning Systemscited by 0
  2. 2026Boosting Transferable Adversarial Attacks against Deep Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023BERT Lost Patience Won't Be Robust to Adversarial Slowdown
  2. 2025A Survey of Early Exit Deep Neural Networks in NLP
  3. 2025Sampling-aware Adversarial Attacks Against Large Language Models
  4. 2026Exploring Jailbreak Vulnerability Across Depth-Based Dynamic Inference in Large Language Models
  5. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
  6. 2026The Diminishing Returns of Early-Exit Decoding in Modern LLMs
  7. 2025OET: Optimization-based prompt injection Evaluation Toolkit
  8. 2025Attack and defense techniques in large language models: A survey and new perspectives