mlsanityData to 14 Oct 2026

Accelerating OpenPangu Inference on NPU via Speculative Decoding

arXiv 2603.03383 ↗First posted 3 Mar 2026cs.DCcited by 1

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

To mitigate the Memory Wall bottleneck encountered by Large Language Models (LLMs) during inference on NPU hardware, and addressing the scarcity of native support for mainstream speculative decoding algorithms on domestic infrastructure, this study presents an end-to-end speculative inference acceleration scheme for OpenPangu-7B.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 19 of its 23 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

19 of 23 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  2. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  3. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  4. 2023Accelerating LLM Inference with Staged Speculative Decoding8 Aug
  5. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  6. 2023Speculative Decoding with Big Little Decoder15 Feb
  7. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  8. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 11

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  7. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  8. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  9. 2022Efficiently Scaling Transformer Inferencecited by 539
  10. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
  11. 2025Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognitioncited by 62

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Efficient Mixture-of-Experts LLM Inference with Apple Silicon NPUscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting
  2. 2024Dovetail: A CPU/GPU Heterogeneous Speculative Decoding for LLM inference
  3. 2026SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
  4. 2024HADES: Hardware Accelerated Decoding for Efficient Speculation in Large Language Models
  5. 2025Speculative Decoding: Performance or Illusion?
  6. 2024Speculative Streaming: Fast LLM Inference without Auxiliary Models
  7. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding
  8. 2026EAGLE-Pangu: Accelerator-Safe Tree Speculative Decoding on Ascend NPUs