mlsanityData to 7 Oct 2026

Approximately Aligned Decoding

arXiv 2410.01103 ↗First posted 1 Oct 2024cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

It is common to reject undesired outputs of Large Language Models (LLMs); however, current methods to do so require an excessive amount of computation to re-sample after a rejection, or distort the distribution of outputs by constraining the output to highly improbable tokens. We present a method, Approximately Aligned Decoding (AprAD), to balance the distortion of the output distribution with computational efficiency, inspired by algorithms from the speculative decoding literature. AprAD allows for the generation of long sequences of text with difficult-to-satisfy constraints, while amplifying low probability outputs much less compared to existing methods. We show through a series of experiments that the task-specific performance of AprAD is comparable to methods that do not distort the output distribution, while being much more computationally efficient.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 18 of its 32 references that are in the map. First posted on arXiv 1 Oct 2024.

Builds on

18 of 32 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  3. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  4. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 564
  5. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  6. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
  7. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  8. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 362
  9. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 359
  10. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 186
  11. 2024Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generationcited by 99
  12. 2023Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programscited by 79
6 more
  1. 2023Tractable Control for Autoregressive Language Generationcited by 65
  2. 2024Grammar-Aligned Decodingcited by 54
  3. 2024Adaptable Logical Control for Large Language Modelscited by 30
  4. 2022Syntax-Aware On-the-Fly Code Completioncited by 10
  5. 2023Most Language Models can be Poets too: An AI Writing Assistant and Constrained Text Generation Studiocited by 9
  6. 2024Token Alignment via Character Matching for Subword Completioncited by 9

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025CRANE: Reasoning with constrained LLM generationcited by 38
  2. 2025Speeding up Speculative Decoding via Sequential Approximate Verificationcited by 11
  3. 2025Constrained Adaptive Rejection Samplingcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification
  2. 2025GRIFFIN: Effective Token Alignment for Faster Speculative Decoding
  3. 2024Fast Best-of-N Decoding via Speculative Rejection
  4. 2024Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inference
  5. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
  6. 2025Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment
  7. 2024Nudging: Inference-time Alignment of LLMs via Guided Decoding
  8. 2025Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding