mlsanityData to 13 Oct 2026

SelfJudge: Faster Speculative Decoding via Self-Supervised Judge Verification

arXiv 2510.02329 ↗First posted 26 Sep 2025cs.CL cs.AIcited by 4

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Speculative decoding accelerates LLM inference by verifying candidate tokens from a draft model against a larger target model. Recent judge decoding boosts this process by relaxing verification criteria by accepting draft tokens that may exhibit minor discrepancies from target model output, but existing methods are restricted by their reliance on human annotations or tasks with verifiable ground truths, limiting generalizability across diverse NLP tasks. We propose SelfJudge, which trains judge verifiers via self-supervision of the target model. Our method measures semantic preservation by assessing whether token-substituted responses preserve the meaning of original responses, enabling automatic verifier training across diverse NLP tasks. Our experiments show SelfJudge achieves superior inference-accuracy trade-offs than judge decoding baselines, offering a broadly applicable solution for faster LLM inference.

Key facts. Cited by 4 papers in the map (data through 13 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 30 of its 33 references that are in the map. First posted on arXiv 26 Sep 2025.

Builds on

30 of 33 references.

Closest ancestors · 18

Same direction, newest first

  1. 2025AutoJudge: Judge Decoding Without Manual Annotation28 Apr
  2. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
  3. 2025Towards Optimal Multi-draft Speculative Decoding26 Feb
  4. 2025Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment31 Jan
  5. 2024SuffixDecoding: Extreme Speculative Decoding for Emerging AI Applications7 Nov
  6. 2024Faster Cascades via Speculative Decoding29 May
  7. 2024TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding18 Apr
  8. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  9. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  10. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  11. 2023REST: Retrieval-Based Speculative Decoding14 Nov
  12. 2023SpecTr: Fast Speculative Decoding via Optimal Transport23 Oct
6 more
  1. 2023DistillSpec: Improving Speculative Decoding via Knowledge Distillation12 Oct
  2. 2023Online Speculative Decoding11 Oct
  3. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  4. 2023Speculative Decoding with Big Little Decoder15 Feb
  5. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  6. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,876
  4. 2020Scaling Laws for Neural Language Modelscited by 6,494
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,951
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,799
  7. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,313
  8. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 810
  9. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 457
  10. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 404
  11. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 319
  12. 2024Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recyclingcited by 47

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Why and When Visual Token Pruning Fails? A Study on Relevant Visual Information Shift in MLLMs Decodingcited by 6
  2. 2026JuDi: Revisiting Judge Decoding from First Principles via Training-Free Distributional Divergencecited by 2
  3. 2026Training-free Dropout Sampling for Semantic Token Acceptance in Speculative Decodingcited by 0
  4. 2026Beyond the Target: From Imitation to Collaboration in Speculative Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration
  2. 2024Accelerating Large Language Model Inference with Self-Supervised Early Exits
  3. 2025Think Before You Accept: Semantic Reflective Verification for Faster Speculative Decoding
  4. 2023Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding
  5. 2026SEED: Self-Speculative Decoding via Implicit Encoder-Decoder
  6. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
  7. 2025Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling
  8. 2025Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match