mlsanityData to 14 Oct 2026

One Jump Is All You Need: Short-Cutting Transformers for Early Exit Prediction with One Jump to Fit All Exit Levels

arXiv 2504.13984 ↗First posted 18 Apr 2025cs.LG cs.AI cs.CLcited by 0

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

To reduce the time and computational costs of inference of large language models, there has been interest in parameter-efficient low-rank early-exit casting of transformer hidden-representations to final-representations. Such low-rank short-cutting has been shown to outperform identity shortcuts at early model stages while offering parameter-efficiency in shortcut jumps. However, current low-rank methods maintain a separate early-exit shortcut jump to final-representations for each transformer intermediate block-level during inference. In this work, we propose selection of a single One-Jump-Fits-All (OJFA) low-rank shortcut that offers over a 30x reduction in shortcut parameter costs during inference. We show that despite this extreme reduction, our OJFA choice largely matches the performance of maintaining multiple shortcut jumps during inference and offers stable precision from all transformer block-levels for GPT2-XL, Phi3-Mini and Llama2-7B transformer models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 9 of its 10 references that are in the map. First posted on arXiv 18 Apr 2025.

Builds on

9 of 10 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Normalized Narrow Jump To Conclusions: Normalized Narrow Shortcuts for Parameter Efficient Early Exit Transformer Prediction21 Sep

Other references · 8

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  6. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  7. 2020The Right Tool for the Job: Matching Model and Instance Complexitiescited by 126
  8. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 90

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026The Diminishing Returns of Early-Exit Decoding in Modern LLMs
  2. 2026LEAP: Layer-wise Exit-Aware Pretraining for Efficient Transformer Inference
  3. 2024Early Exit Is a Natural Capability in Transformer-based Models: An Empirical Study on Early Exit without Joint Optimization
  4. 2024An Efficient Inference Framework for Early-exit Large Language Models
  5. 2024RAEE: A Robust Retrieval-Augmented Early Exit Framework for Efficient Inference
  6. 2026LRCC: Generalizing Low-Rank Compression with Conditional Computation
  7. 2024Accelerating Large Language Model Inference with Self-Supervised Early Exits
  8. 2025Improving Model Representation and Reducing KV Cache via Skip Connections with First Value Heads