mlsanityData to 7 Oct 2026

EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models

arXiv 2402.00518 ↗First posted 1 Feb 2024cs.LG cs.AI cs.CLcited by 13

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

This work introduces EE-Tuning, a lightweight and economical solution to training/tuning early-exit large language models (LLMs). In contrast to the common approach of full-parameter pre-training, EE-Tuning augments any pre-trained (and possibly fine-tuned) standard LLM with additional early-exit layers that are tuned in a parameter-efficient manner, which requires significantly less computational resources and training data. Our implementation of EE-Tuning achieves outstanding training efficiency via extensive performance optimizations, as well as scalability due to its full compatibility with 3D parallelism. Results of systematic experiments validate the efficacy of EE-Tuning, confirming that effective early-exit LLM inference can be achieved with a limited training budget. In hope of making early-exit LLMs accessible to the community, we release the source code of our implementation of EE-Tuning at https://github.com/pan-x-c/EE-LLM.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 46 of its 55 references that are in the map. First posted on arXiv 1 Feb 2024.

Builds on

46 of 55 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving8 Dec
  2. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism8 Dec
  3. 2023SmartBERT: A Promotion of Dynamic Early Exiting Mechanism for Accelerating BERT Inference16 Mar
  4. 2022Confident Adaptive Language Modeling14 Jul
  5. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions9 Jun
  6. 2021Accelerating BERT Inference for Sequence Labeling via Early-Exit28 May

Other references · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2016Layer Normalizationcited by 5,579
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  11. 2019Root Mean Square Layer Normalizationcited by 1,425
  12. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
28 more
  1. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  2. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  3. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  4. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  5. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  6. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  7. 2021Dynamic Neural Networks: A Surveycited by 405
  8. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
  9. 2022Reducing Activation Recomputation in Large Transformer Modelscited by 300
  10. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 267
  11. 2016Using the Output Embedding to Improve Language Modelscited by 266
  12. 2020DynaBERT: Dynamic BERT with Adaptive Width and Depthcited by 246
  13. 2019Depth-Adaptive Transformercited by 227
  14. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 224
  15. 2020The Right Tool for the Job: Matching Model and Instance Complexitiescited by 125
  16. 2023SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inferencecited by 89
  17. 2023Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingcited by 80
  18. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 73
  19. 2020Why should we add early exits to neural networks?cited by 70
  20. 2015Conditional Deep Learning for Energy-Efficient and Enhanced Pattern Recognitioncited by 67
  21. 2023Data-Juicer: A One-Stop Data Processing System for Large Language Modelscited by 52
  22. 2023The Benefits of Bad Advice: Autocontrastive Decoding across Model Layerscited by 28
  23. 2022A Survey on Dynamic Neural Networks for Natural Language Processingcited by 24
  24. 2020ELF: An Early-Exiting Framework for Long-Tailed Classificationcited by 20
  25. 2023An Empirical Study of Instruction-tuning Large Language Models in Chinesecited by 17
  26. 2023Jointly-Learned Exit and Inference for a Dynamic Neural Network : JEI-DNNcited by 15
  27. 2023AdaDiff: Accelerating Diffusion Models through Step-Wise Adaptive Computationcited by 14
  28. 2021Improving the Accuracy of Early Exits in Multi-Exit Architectures via Curriculum Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 5; 2026: 3.

13 papers.

Sort:

By area:

  1. 2025AI Flow: Perspectives, Scenarios, and Approachescited by 44
  2. 2024Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compressioncited by 34
  3. 2024Fast yet Safe: Early-Exiting with Risk Controlcited by 27
  4. 2024CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud-Edge Collaborationcited by 25
  5. 2024LiveMind: Low-latency Large Language Models with Simultaneous Inferencecited by 10
  6. 2025HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Servingcited by 8
  7. 2024Duo-LLM: A Framework for Studying Adaptive Computation in Large Language Modelscited by 6
  8. 2025Zero Token-Driven Deep Thinking in LLMs: Unlocking the Full Potential of Existing Parameters via Cyclic Refinementcited by 5
  9. 2026From Prefix Cache to Fusion RAG Cache: Accelerating LLM Inference in Retrieval-Augmented Generationcited by 5
  10. 2025Dynamic Rebatching for Efficient Early-Exit Inference with DREXcited by 2
  11. 2026The Shape of Overthinking: Backtracking Bursts in Long Reasoning Tracescited by 2
  12. 2025ACME: Adaptive Customization of Large Models via Distributed Systemscited by 1
1 more
  1. 2026Off-Axis, On Purpose: Where a Transformer Computes Concepts and Why it Does Socited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024An Efficient Inference Framework for Early-exit Large Language Models
  2. 2023Towards Better Parameter-Efficient Fine-Tuning for Large Language Models: A Position Paper
  3. 2026The Diminishing Returns of Early-Exit Decoding in Modern LLMs
  4. 2021CPM-2: Large-scale Cost-effective Pre-trained Language Models
  5. 2023Tuning LayerNorm in Attention: Towards Efficient Multi-Modal LLM Finetuning
  6. 2024Outlier-weighed Layerwise Sampling for LLM Fine-tuning
  7. 2025Towards Fast LLM Fine-tuning through Zeroth-Order Optimization with Projected Gradient-Aligned Perturbations
  8. 2025Optimal Singular Damage: Efficient LLM Inference in Low Storage Regimes