mlsanityData to 7 Oct 2026

BitSkip: An Empirical Analysis of Quantization and Early Exit Composition in Transformers

arXiv 2510.23766 ↗First posted 27 Oct 2025cs.CLcited by 1

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

The pursuit of efficient Large Language Models (LLMs) has led to increasingly complex techniques like extreme quantization and dynamic routing. While individual benefits of these methods are well-documented, their compositional effects remain poorly understood. This paper introduces BitSkip, a hybrid architectural framework for systematically exploring these interactions. Counter-intuitively, our findings reveal that a simple 8-bit quantized model without Hadamard transform (BitSkip-V1) not only outperforms its more complex 4-bit and Hadamard-enhanced counterparts but also competes the full-precision baseline in quality (perplexity of 1.13 vs 1.19) . The introduction of Hadamard transforms, even at 8-bit precision, catastrophically degraded performance by over 37,000%, tracing fundamental training instability. Our BitSkip-V1 recipe demonstrates superior early-exit characteristics, with layer 18 providing optimal 32.5% speed gain for minimal 4% quality loss.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 27 of its 30 references that are in the map. First posted on arXiv 27 Oct 2025.

Builds on

27 of 30 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding25 Apr
  2. 2022Confident Adaptive Language Modeling14 Jul

Other references · 25

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  3. 2016Pointer Sentinel Mixture Modelscited by 2,992
  4. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  7. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  8. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  10. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  11. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  12. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
13 more
  1. 2021A White Paper on Neural Network Quantizationcited by 453
  2. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  3. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  4. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
  5. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
  6. 2019Depth-Adaptive Transformercited by 227
  7. 2023Efficient Large Language Models: A Surveycited by 183
  8. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 175
  9. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 162
  10. 2022Compute-Efficient Deep Learning: Algorithmic Trends and Opportunitiescited by 38
  11. 2025BitNet b1.58 2B4T Technical Reportcited by 30
  12. 2025BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMscited by 16
  13. 2025DEL: Context-Aware Dynamic Exit Layer for Efficient Self-Speculative Decodingcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Quantizing Recursive Reasoning Modelscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Quantizing Looped Transformers: Feedback Exposure and Calibration Blindness
  2. 2026Revisiting the Shape Convention of Transformer Language Models
  3. 2024Early Exit Is a Natural Capability in Transformer-based Models: An Empirical Study on Early Exit without Joint Optimization
  4. 2026QuEPT: Quantized Elastic Precision Transformers with One-Shot Calibration for Multi-Bit Switching
  5. 2026A KL Lens on Quantization: Fast, Forward-Only Sensitivity for Mixed-Precision SSM-Transformer Models
  6. 2024MLKV: Multi-Layer Key-Value Heads for Memory Efficient Transformer Decoding
  7. 2025Pimba: A Processing-in-Memory Acceleration for Post-Transformer Large Language Model Serving
  8. 2024Do Efficient Transformers Really Save Computation?