mlsanityData to 14 Oct 2026

LP-Spec: Leveraging LPDDR PIM for Efficient LLM Mobile Speculative Inference with Architecture-Dataflow Co-Optimization

arXiv 2508.07227 ↗First posted 10 Aug 2025cs.ARcited by 9

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

LLM inference on mobile devices faces extraneous challenges due to limited memory bandwidth and computational resources. To address these issues, speculative inference and processing-in-memory (PIM) techniques have been explored at the algorithmic and hardware levels. However, speculative inference results in more compute-intensive GEMM operations, creating new design trade-offs for existing GEMV-accelerated PIM architectures. Furthermore, there exists a significant amount of redundant draft tokens in tree-based speculative inference, necessitating efficient token management schemes to minimize energy consumption. In this work, we present LP-Spec, an architecture-dataflow co-design leveraging hybrid LPDDR5 performance-enhanced PIM architecture with draft token pruning and dynamic workload scheduling to accelerate LLM speculative inference. A near-data memory controller is proposed to enable data reallocation between DRAM and PIM banks. Furthermore, a data allocation unit based on the hardware-aware draft token pruner is developed to minimize energy consumption and fully exploit parallel execution opportunities. Compared to end-to-end LLM inference on other mobile solutions such as mobile NPUs or GEMV-accelerated PIMs, our LP-Spec achieves 13.21x, 7.56x, and 99.87x improvements in performance, energy efficiency, and energy-delay-product (EDP). Compared with prior AttAcc PIM and RTX 3090 GPU, LP-Spec can obtain 12.83x and 415.31x EDP reduction benefits.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 19 of its 39 references that are in the map. First posted on arXiv 10 Aug 2025.

Builds on

19 of 39 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025PAPI: Exploiting Dynamic Parallelism in Large Language Model Decoding with a Processing-In-Memory-Enabled Computing System21 Feb
  2. 2024ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding21 Feb
  3. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  4. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  5. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  6. 2022Fast Inference from Transformers via Speculative Decoding30 Nov
  7. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation30 Mar

Other references · 12

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  7. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  8. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  9. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  10. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  11. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  12. 2024NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencingcited by 52

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 7.

9 papers.

  1. 2025P3-LLM: An Integrated NPU-PIM Accelerator for Edge LLM Inference Using Hybrid Numerical Formatscited by 3
  2. 2026Parameter-Free Encoders Remain Viable for RDB Foundation Modelscited by 2
  3. 2025DCC: Data-Centric Compilation of Machine Learning Kernels for Processing-In-Memory Architecturescited by 0
  4. 2026A Scheduling Framework for Efficient MoE Inference on Edge GPU-NDP Systemscited by 0
  5. 2026ELMoE-3D: Leveraging Intrinsic Elasticity of MoE for Hybrid-Bonding-Enabled Self-Speculative Decoding in On-Premises Servingcited by 0
  6. 2026AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Servingcited by 0
  7. 2026TokenStack: A Heterogeneous HBM-PIM Architecture and Runtime for Efficient LLM Inferencecited by 0
  8. 2026Cassandra: Enabling Reasoning LLMs at Edge via Self-Speculative Decodingcited by 0
  9. 2026SPECTRA: Adaptive Execution of Speculative Decoding on a Runtime-Reconfigurable Tiled Architecturecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LeanStream: A Speculate-and-Refine Streaming Framework for Efficient on-Device LLM Inference
  2. 2026Compiler-Assisted Speculative Sampling for Accelerated LLM Inference on Heterogeneous Edge Devices
  3. 2026Lever: Speculative LLM Inference on Smartphones
  4. 2025MVDRAM: Enabling GeMV Execution in Unmodified DRAM for Low-Bit LLM Acceleration
  5. 2026Hydra: Phase-Aware Workload Characterization of LLM Inference across Edge SoC Generations, Backends, and Quantization Levels
  6. 2025Characterizing Mobile SoC for Accelerating Heterogeneous LLM Inference
  7. 2026When NPUs Are Not Always Faster: A Stage-Level Analysis of Mobile LLM Inference
  8. 2025Ghidorah: Fast LLM Inference on Edge with Speculative Decoding and Hetero-Core Parallelism