# Component-Aware Self-Speculative Decoding in Hybrid Language Models

arXiv: https://arxiv.org/abs/2605.01106 · HTML: https://mlsanity.com/paper/2605.01106/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 21 of its 24 references that are in the map.
- First posted on arXiv 1 May 2026.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 21 of 21)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [Gated Linear Attention Transformers with Hardware-Efficient Training](https://mlsanity.com/paper/2312.06635/) (2023)
- [Jamba: A Hybrid Transformer-Mamba Language Model](https://mlsanity.com/paper/2403.19887/) (2024)
- [LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding](https://mlsanity.com/paper/2404.16710/) (2024)
- [Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling](https://mlsanity.com/paper/2406.07522/) (2024)
- [SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration](https://mlsanity.com/paper/2410.06916/) (2024)
- [Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance](https://mlsanity.com/paper/2507.22448/) (2025)
- [CLaSp: In-Context Layer Skip for Self-Speculative Decoding](https://mlsanity.com/paper/2505.24196/) (2025)
- [STree: Speculative Tree Decoding for Hybrid State-Space Models](https://mlsanity.com/paper/2505.14969/) (2025)
- [RAD: Redundancy-Aware Distillation for Hybrid Models via Self-Speculative Decoding](https://mlsanity.com/paper/2505.22135/) (2025)
- [Exploring the Limitations of Mamba in COPY and CoT Reasoning](https://mlsanity.com/paper/2410.03810/) (2024)
- [KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem](https://mlsanity.com/paper/2602.20217/) (2026)
- [Component Ablation for Efficient Hybrid Language Model Architectures: Performance, Resilience, and Compression Implications](https://mlsanity.com/paper/2603.22473/) (2026)
- [Mamba Drafters for Speculative Decoding](https://mlsanity.com/paper/2506.01206/) (2025)
- [ConfLayers: Adaptive Confidence-based Layer Skipping for Self-Speculative Decoding](https://mlsanity.com/paper/2604.14612/) (2026)

## Built on by (most cited first, 1 of 1)

- [LumoTree: Path-Parallel Speculative Verification for Hybrid Language Models](https://mlsanity.com/paper/2609.23900/) (2026)

## Abstract (arXiv)

Speculative decoding accelerates autoregressive inference by drafting candidate tokens with a fast model and verifying them in parallel with the target. Self-speculative methods avoid the need for an external drafter but have been studied exclusively in homogeneous Transformer architectures. We introduce component-aware self-speculative decoding, the first method to exploit the internal architectural heterogeneity of hybrid language models, isolating the SSM/linear-attention subgraph as a zero-cost internal draft. We evaluate this on two architecturally distinct hybrid families: Falcon-H1 (parallel: Mamba-2 + attention per layer) and Qwen3.5 (sequential: interleaved linear and attention layers), with a pure Transformer control (Qwen2.5). Parallel hybrids achieve acceptance rates of alpha = 0.68 at draft length k=2 under greedy decoding, while sequential hybrids yield only alpha = 0.038 -- an 18x gap attributable to how each architecture integrates its components. The property is scale-invariant: Falcon-H1 at 3B reproduces the rates observed at 0.5B. We further show that perplexity degradation from a companion ablation study predicts speculative viability without running speculative decoding: a 3.15x ratio (Falcon) maps to alpha = 0.37 at k=4, while 81.96x (Qwen) maps to alpha = 0.019. For sequential hybrids, generic LayerSkip achieves 12x higher acceptance rates than the component-aware strategy. The composition pattern of hybrid models -- not merely the presence of alternative components -- determines whether component-level self-speculation is viable.