mlsanityData to 7 Oct 2026

Parser States Already Know: Structure-Conditioned KV Persistence for Structured Generation

arXiv 2608.28276 ↗First posted 28 Aug 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Structured generation underpins large language model (LLM) agents that produce JSON, SQL, and function calls, where a single wrong field can cause the downstream action to fail. Constrained decoding already tracks parser transitions to enforce formal validity, and these transitions expose how generated tokens participate in schema-critical decisions such as required fields, arguments, and structural boundaries under the active grammar. Existing KV compression largely leaves this task-relevant structural signal unused. We introduce PASK (Parser-Aware Structural KV Persistence), which turns parser-derived structure into layer-group-specific KV persistence decisions. PASK addresses the mismatch between model-side KV sensitivity and task-level structured risk by using task-error sensitivity to set minimum protection floors and attention-output distortion to allocate residual KV capacity. An offline calibration stage compiles these signals into a persistence policy, leaving only lightweight structure-conditioned lookup online. At a targe total KV budget of 0.33, PASK outperforms the strongest compressed baseline by 17.39 percentage points on average across eight BFCL non-live and Live subcategories on Qwen3-4B. In end-to-end serving, PASK achieves up to 2.2x higher throughput and 3.3x lower TPOT, while using 0.53x the peak GPU memory of Full KV.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 34 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

27 of 34 references.

References · 27

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  3. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  4. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  5. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  6. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  7. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 466
  8. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 428
  9. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  10. 2024Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generationcited by 99
  11. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inferencecited by 57
  12. 2024DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMscited by 42
15 more
  1. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compressioncited by 41
  2. 2024IterGen: Iterative Semantic-aware Structured LLM Generation with Backtrackingcited by 28
  3. 2025CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspectivecited by 28
  4. 2026Fast KV Compaction via Attention Matchingcited by 28
  5. 2024SCOPE: Optimizing Key-Value Cache Compression in Long-context Generationcited by 20
  6. 2025FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Accelerationcited by 13
  7. 2026Don't Waste Bits! Adaptive KV-Cache Quantization for Lightweight On-Device LLMscited by 7
  8. 2026XGrammar-2: Dynamic and Efficient Structured Generation Engine for Agentic LLMscited by 6
  9. 2026IntentKV: Cross-Turn Intent-Aware KV Cache Pruning for Agent Inferencecited by 6
  10. 2026StructKV: Preserving the Structural Skeleton for Scalable Long-Context Inferencecited by 3
  11. 2026TriAxialKV: Toward Extreme Low-Precision KV-Cache Quantization for Agentic Inference Taskscited by 3
  12. 2026ArborKV: Structure-Aware KV Cache Management for Scaling Tree-based LLM Reasoningcited by 3
  13. 2026CodeComp: Structural KV Cache Compression for Agentic Codingcited by 1
  14. 2026Transactional Attention: Semantic Sponsorship for KV-Cache Retentioncited by 1
  15. 2026Adaptive Filtering of the KV Cache: Diagnosing and Correcting Structural-Role Bias in LLM Inferencecited by 1

Built on by

Not cited yet.