mlsanityData to 7 Oct 2026

Breaking the Autoregressive Chain: Hyper-Parallel Decoding for Efficient LLM-Based Attribute Value Extraction

arXiv 2604.26209 ↗First posted 29 Apr 2026cs.CL cs.AIcited by 1

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Some text generation tasks, such as Attribute Value Extraction (AVE), require decoding multiple independent sequences from the same document context. While standard autoregressive decoding is slow due to its sequential nature, the independence between output sequences offers an opportunity for parallelism. We present Hyper-Parallel Decoding, a novel decoding algorithm that accelerates offline decoding by leveraging both shared memory and computation across batches. HPD enables out-of-order token generation through position ID manipulation, significantly improving efficiency. Experiments on AVE show that attribute-value pairs are conditionally independent, enabling us to parallelize value generation within each prompt. By further stacking multiple documents within a single prompt, we can decode in parallel up to 96 tokens per prompt. HPD works with all LLMs, and reduces both inference costs and total inference time by up to 13.8X without compromising output quality, potentially saving hundreds of thousands of dollars on industry AVE tasks. Although designed for attribute extraction, HPD makes no assumptions unique to the AVE domain and can in theory be applied to other scenarios with independent output structures.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 22 of its 44 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

22 of 44 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  2. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  3. 2024APAR: LLMs Can Do Auto-Parallel Auto-Regressive Decoding12 Jan
  4. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  3. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  4. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 544
  5. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
  6. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 180
  7. 2023Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generationcited by 103
  8. 2023Batch Prompting: Efficient Inference with Large Language Model APIscited by 84
  9. 2021MAVE: A Product Dataset for Multi-source Attribute Value Extractioncited by 53
  10. 2022OA-Mine: Open-World Attribute Mining for E-Commerce Products with Weak Supervisioncited by 20
  11. 2023ExtractGPT: Exploring the Potential of Large Language Models for Product Attribute Value Extractioncited by 20
  12. 2024Using LLMs for the Extraction and Normalization of Product Attribute Valuescited by 20
6 more
  1. 2024ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extractioncited by 17
  2. 2021Answering Product-Questions by Utilizing Questions from Other Contextually Similar Productscited by 12
  3. 2023A Unified Generative Approach to Product Attribute-Value Identificationcited by 10
  4. 2021Graph-based Multilingual Product Retrieval in E-commerce Searchcited by 9
  5. 2021End-to-End Conversational Search for Online Shopping with Utterance Transfercited by 7
  6. 2024An Empirical Comparison of Generative Approaches for Product Attribute-Value Identificationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Scaling E-Commerce Attribute Extraction with Parallel Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMs
  2. 2026Efficient Document Parsing via Parallel Token Prediction
  3. 2026HPD-Parsing: Hierarchical Parallel Document Parsing
  4. 2024Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding
  5. 2025ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMs
  6. 2024ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding
  7. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs
  8. 2026Beyond Truncation: Rethinking LLM Decoding as Ensemble Pruning