mlsanityData to 7 Oct 2026

Multi-Exit Vision Transformer for Dynamic Inference

arXiv 2106.15183 ↗First posted 29 Jun 2021cs.CVcited by 28

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Deep neural networks can be converted to multi-exit architectures by inserting early exit branches after some of their intermediate layers. This allows their inference process to become dynamic, which is useful for time critical IoT applications with stringent latency requirements, but with time-variant communication and computation resources. In particular, in edge computing systems and IoT networks where the exact computation time budget is variable and not known beforehand. Vision Transformer is a recently proposed architecture which has since found many applications across various domains of computer vision. In this work, we propose seven different architectures for early exit branches that can be used for dynamic inference in Vision Transformer backbones. Through extensive experiments involving both classification and regression problems, we show that each one of our proposed architectures could prove useful in the trade-off between accuracy and speed.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 24 of its 32 references that are in the map. First posted on arXiv 29 Jun 2021.

Builds on

24 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Single-Layer Vision Transformers for More Accurate Early Exits with Less Overhead19 May

Other references · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2021Vision Transformers for Dense Predictioncited by 1,789
  6. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  7. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  8. 2021Transformers in Vision: A Surveycited by 1,009
  9. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  10. 2021Dynamic Neural Networks: A Surveycited by 405
  11. 2021Pay Attention to MLPscited by 367
  12. 2021FNet: Mixing Tokens with Fourier Transformscited by 343
11 more
  1. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  2. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 285
  3. 2020Why should we add early exits to neural networks?cited by 70
  4. 2021TransCrowd: weakly-supervised crowd counting with transformerscited by 61
  5. 2020Triple Wins: Boosting Accuracy, Robustness and Efficiency Together by Enabling Input-Adaptive Inferencecited by 45
  6. 2020CNN-based Density Estimation and Crowd Counting: A Surveycited by 43
  7. 2020Ambient Sound Helps: Audiovisual Crowd Counting in Extreme Conditionscited by 13
  8. 2019Dense Scale Network for Crowd Countingcited by 12
  9. 2021Improving the Accuracy of Early Exits in Multi-Exit Architectures via Curriculum Learningcited by 9
  10. 2021Can Attention Enable MLPs To Catch Up With CNNs?cited by 7
  11. 2021Temporal Early Exits for Efficient Video Object Detectioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 1; 2024: 5; 2025: 8; 2026: 8.

28 papers.

Sort:

By area:

  1. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  2. 2023LGViT: Dynamic Early Exiting for Accelerating Vision Transformercited by 30
  3. 2021Improving Transformers with Probabilistic Attention Keyscited by 28
  4. 2025Similarity-Guided Layer-Adaptive Vision Transformer for UAV Trackingcited by 22
  5. 2022Dynamic Split Computing for Efficient Deep Edge Intelligencecited by 18
  6. 2022Continual Transformers: Redundancy-Free Attention for Online Inferencecited by 12
  7. 2025Token Pruning in Audio Transformers: Optimizing Performance and Decoding Patch Importancecited by 12
  8. 2022Multi-Tailed Vision Transformer for Efficient Inferencecited by 10
  9. 2022Efficient High-Resolution Deep Learning: A Surveycited by 9
  10. 2025HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Servingcited by 8
  11. 2025ThinkingViT: Matryoshka Thinking Vision Transformer for Elastic Inferencecited by 6
  12. 2026ViTNT-FIQA: Training-Free Face Image Quality Assessment with Vision Transformerscited by 4
16 more
  1. 2024Layer-wise Alignment: Examining Safety Alignment Across Image Encoder Layers in Vision Language Modelscited by 3
  2. 2025You Only Look Once at Anytime (AnytimeYOLO): Analysis and Optimization of Early-Exits for Object-Detectioncited by 3
  3. 2024EERO: Early Exit with Reject Option for Efficient Classification with limited budgetcited by 2
  4. 2025Image Recognition with Online Lightweight Vision Transformer: A Surveycited by 2
  5. 2025Efficiency Robustness of Dynamic Deep Learning Systemscited by 1
  6. 2025ACME: Adaptive Customization of Large Models via Distributed Systemscited by 1
  7. 2026Layer-Guided UAV Tracking: Enhancing Efficiency and Occlusion Robustnesscited by 1
  8. 2022Design and Prototyping Distributed CNN Inference Acceleration in Edge Computingcited by 0
  9. 2024Image-Specific Adaptation of Transformer Encoders for Compute-Efficient Segmentationcited by 0
  10. 2024Learning Motion Blur Robust Vision Transformers for Real-Time UAV Trackingcited by 0
  11. 2026A Comparative Study of CNN Optimization Methods for Edge AI: Exploring the Role of Early Exitscited by 0
  12. 2026EX-FIQA: Leveraging Intermediate Early eXit Representations from Vision Transformers for Face Image Quality Assessmentcited by 0
  13. 2026Progressive Approximation in Deep Residual Networks: Theory and Validationcited by 0
  14. 2026Amortized-Precision Quantization for Early-Exit Vision Transformerscited by 0
  15. 2026The Rescue Effect: Spatio-Semantic Early Exit Bypasses Quantization Collapse in CLIPcited by 0
  16. 2026ViT-FREE: Efficient Face Recognition via Early Exiting and Synthetic Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Janus: Collaborative Vision Transformer Under Dynamic Network Environment
  2. 2025Rethinking Vision Transformer Depth via Structural Reparameterization
  3. 2022You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model
  4. 2025ToFe: Lagged Token Freezing and Reusing for Efficient Vision Transformer Inference
  5. 2023Dynamic Perceiver for Efficient Visual Recognition
  6. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformer
  7. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference
  8. 2025EdgeFlex-Transformer: Transformer Inference for Edge Devices