# Breaking the Autoregressive Chain: Hyper-Parallel Decoding for Efficient LLM-Based Attribute Value Extraction

arXiv: https://arxiv.org/abs/2604.26209 · HTML: https://mlsanity.com/paper/2604.26209/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 29 Apr 2026.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [Fast Transformer Decoding: One Write-Head is All You Need](https://mlsanity.com/paper/1911.02150/) (2019)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Break the Sequential Dependency of LLM Inference Using Lookahead Decoding](https://mlsanity.com/paper/2402.02057/) (2024)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [Flex Attention: A Programming Model for Generating Optimized Attention Kernels](https://mlsanity.com/paper/2412.05496/) (2024)
- [Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation](https://mlsanity.com/paper/2307.15337/) (2023)
- [Batch Prompting: Efficient Inference with Large Language Model APIs](https://mlsanity.com/paper/2301.08721/) (2023)
- [MAVE: A Product Dataset for Multi-source Attribute Value Extraction](https://mlsanity.com/paper/2112.08663/) (2021)
- [APAR: LLMs Can Do Auto-Parallel Auto-Regressive Decoding](https://mlsanity.com/paper/2401.06761/) (2024)
- [OA-Mine: Open-World Attribute Mining for E-Commerce Products with Weak Supervision](https://mlsanity.com/paper/2204.13874/) (2022)
- [ExtractGPT: Exploring the Potential of Large Language Models for Product Attribute Value Extraction](https://mlsanity.com/paper/2310.12537/) (2023)
- [Using LLMs for the Extraction and Normalization of Product Attribute Values](https://mlsanity.com/paper/2403.02130/) (2024)
- [ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extraction](https://mlsanity.com/paper/2404.15592/) (2024)
- [Answering Product-Questions by Utilizing Questions from Other Contextually Similar Products](https://mlsanity.com/paper/2105.08956/) (2021)
- [A Unified Generative Approach to Product Attribute-Value Identification](https://mlsanity.com/paper/2306.05605/) (2023)
- [Graph-based Multilingual Product Retrieval in E-commerce Search](https://mlsanity.com/paper/2105.02978/) (2021)
- [End-to-End Conversational Search for Online Shopping with Utterance Transfer](https://mlsanity.com/paper/2109.05460/) (2021)
- [An Empirical Comparison of Generative Approaches for Product Attribute-Value Identification](https://mlsanity.com/paper/2407.01137/) (2024)

## Built on by (most cited first, 1 of 1)

- [Scaling E-Commerce Attribute Extraction with Parallel Decoding](https://mlsanity.com/paper/2609.09716/) (2026)

## Abstract (arXiv)

Some text generation tasks, such as Attribute Value Extraction (AVE), require decoding multiple independent sequences from the same document context. While standard autoregressive decoding is slow due to its sequential nature, the independence between output sequences offers an opportunity for parallelism. We present Hyper-Parallel Decoding, a novel decoding algorithm that accelerates offline decoding by leveraging both shared memory and computation across batches. HPD enables out-of-order token generation through position ID manipulation, significantly improving efficiency. Experiments on AVE show that attribute-value pairs are conditionally independent, enabling us to parallelize value generation within each prompt. By further stacking multiple documents within a single prompt, we can decode in parallel up to 96 tokens per prompt. HPD works with all LLMs, and reduces both inference costs and total inference time by up to 13.8X without compromising output quality, potentially saving hundreds of thousands of dollars on industry AVE tasks. Although designed for attribute extraction, HPD makes no assumptions unique to the AVE domain and can in theory be applied to other scenarios with independent output structures.