mlsanityData to 7 Oct 2026

Early exit and edge inference

215 papers0 this weekLow confidence in this grouping: the papers may belong to more than one topic, or the group may changeLow confidence in this grouping: the papers may belong to more than one topic, or the group may change · About

Keywords: inference, early, exit, early-exit, edge

Key facts. Early exit and edge inference: 215 papers on arXiv since 2021 (data through 7 Oct 2026). 29 of its 215 mapped papers (13%) cite “Confident Adaptive Language Modeling” (2022, arXiv 2207.07061). 29 of its 215 mapped papers (13%) cite “LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding” (2024, arXiv 2404.16710). 5 new papers 24 Sep–7 Oct.

What’s new

24 Sep – 7 Oct 2026

5 new papers, 24 Sep – 7 Oct; 0 this week.

  1. Earlier · 5
  2. TopK-Guided: Adaptive, Budget-Aware Activation Sparsity for Efficient LLM Inference
  3. Sol-H3: Recursive Self-Improvement for MiniMax-H3 Inference Acceleration on Sol-Engine across Cloud and Edge also fits: Fast autoregressive visual generation
  4. SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Serving also fits: Diffusion LLM and MoE inference
  5. When Less Compute Is More: Adaptive Early Exit Improves Pretrained Outlier Detection
  6. Decoupled Early Exits for Task-Dependent Compute Allocation in Flow-Matching VLAs also fits: Visual token pruning in multimodal models

Foundations

Most cited by papers here

  1. Confident Adaptive Language Modeling2022cited by 29 here
  2. LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding2024cited by 29 here
  3. Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions2021cited by 17 here
  4. Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges2021cited by 16 here
  5. EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism2023cited by 12 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2021Accelerating BERT Inference for Sequence Labeling via Early-Exitcited by 10 here
  2. 2022A Simple Hash-Based Early Exiting Approach For Language Understanding and Generationcited by 10 here
  3. 2022Confident Adaptive Language Modelingcited by 29 here
  4. 2023SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inferencecited by 10 here
  5. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 29 here
19 more it builds on, not in this chain
  1. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challengescited by 16 here
  2. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 17 here
  3. 2021Zero Time Waste: Recycling Predictions in Early Exit Neural Networkscited by 11 here
  4. 2022Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inferencecited by 7 here
  5. 2022You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Modelcited by 11 here
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 40 here
  7. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 22 here
  8. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 8 here
  9. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 11 here
  10. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 19 here
  11. 2023SplitEE: Early Exit in Deep Neural Networks with Split Computingcited by 7 here
  12. 2023Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingcited by 16 here
  13. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 12 here
  14. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 8 here
  15. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 15 here
  16. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 10 here
  17. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 12 here
  18. 2024EdgeShard: Efficient LLM Inference via Collaborative Edge Computingcited by 10 here
  19. 2024Fast yet Safe: Early-Exiting with Risk Controlcited by 8 here

The 24 most cited of 230.

Browse

30 of 215, ranked 6–35

  1. 6You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model202211
  2. 7Zero Time Waste: Recycling Predictions in Early Exit Neural Networks202111
  3. 8EdgeShard: Efficient LLM Inference via Collaborative Edge Computing202410
  4. 9A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation202210
  5. 10Accelerating BERT Inference for Sequence Labeling via Early-Exit202110
  6. 11Fast yet Safe: Early-Exiting with Risk Control20248
  7. 12CEEBERT: Cross-Domain Inference in Early Exit BERT20247
  8. 13SplitEE: Early Exit in Deep Neural Networks with Split Computing20237
  9. 14Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference20227
  10. 15Towards Anytime Classification in Early-Exit Architectures by Enforcing Conditional Monotonicity20236
All 30
  1. 16Learning to Weight Samples for Dynamic Early-exiting Networks20226
  2. 17Training-Free Activation Sparsity in Large Language Models20245
  3. 18ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for Accelerating Language Models Inference20235
  4. 19Magic Pyramid: Accelerating Inference with Early Exiting and Token Pruning20215
  5. 20A Survey of Early Exit Deep Neural Networks in NLP20254
  6. 21BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Experts20254
  7. 22Jupiter: Fast and Resource-Efficient Collaborative Inference of Generative LLMs on Edge Devices20254
  8. 23DAdEE: Unsupervised Domain Adaptation in Early Exit PLMs20244
  9. 24SmartBERT: A Promotion of Dynamic Early Exiting Mechanism for Accelerating BERT Inference20234
  10. 25Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving20234
  11. 26HuBERT-EE: Early Exiting HuBERT for Efficient Speech Recognition20224
  12. 27Single-Layer Vision Transformers for More Accurate Early Exits with Less Overhead20214
  13. 28Multi-Exit Vision Transformer for Dynamic Inference20214
  14. 29Elbert: Fast Albert with Confidence-Window Based Early Exit20214
  15. 30PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decoding20263
  16. 31Wanda++: Pruning Large Language Models via Regional Gradients20253
  17. 32A Novel Hat-Shaped Device-Cloud Collaborative Inference Framework for Large Language Models20253
  18. 33Fast and Cost-effective Speculative Edge-Cloud Decoding with Early Exits20253
  19. 34DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving20253
  20. 35EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models20243