mlsanityData to 14 Oct 2026

Lossless Token Sequence Compression via Meta-Tokens

arXiv 2506.00307 ↗First posted 30 May 2025cs.CL cs.AI cs.LGcited by 4

On the map: Efficient LLM inference and serving → Prompt and context compression

Abstract (arXiv)

Existing work on prompt compression for Large Language Models (LLM) focuses on lossy methods that try to maximize the retention of semantic information that is relevant to downstream tasks while significantly reducing the sequence length. In this paper, we introduce a task-agnostic lossless compression technique similar to LZ77 that makes it possible to reduce the input token sequence length on average by 27% and 18% for the two evaluation tasks explored here. Given that we use transformer-based LLMs, this equates to 47% and 33% less encoding computation, respectively, due to the quadratic nature of attention. The token sequence transformation is trivial to reverse and highlights that no semantic information is lost in the process. We evaluate our proposed approach on two tasks that require strict preservation of semantics/syntax and demonstrate that existing lossy compression methods perform poorly in this setting. We find that our lossless compression technique produces only a small gap in performance compared to using the uncompressed input and posit that larger models and an expanded computing budget would likely erase the gap entirely.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 31 of its 40 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

31 of 40 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Prompt Compression for Large Language Models: A Survey16 Oct
  2. 2024TACO-RL: Task Aware Prompt Compression Optimization with Reinforcement Learning19 Sep
  3. 2024AdaComp: Extractive Context Compression with Adaptive Predictor for Retrieval-Augmented Large Language Models3 Sep
  4. 2024Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference2 Sep
  5. 2024500xCompressor: Generalized Prompt Compression for Large Language Models6 Aug
  6. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression19 Mar
  7. 2023TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction24 Oct
  8. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression10 Oct
  9. 2023Discrete Prompt Compression with Reinforcement Learning17 Aug
  10. 2023In-context Autoencoder for Context Compression in a Large Language Model13 Jul

Other references · 21

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  9. 2023Learning to Compress Prompts with Gist Tokenscited by 390
  10. 2023Adapting Language Models to Compress Contextscited by 356
  11. 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 353
  12. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
9 more
  1. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  2. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 287
  3. 2016Using the Output Embedding to Improve Language Modelscited by 272
  4. 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 189
  5. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 99
  6. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 76
  7. 2024Learning to Compress Prompt in Natural Language Formatscited by 55
  8. 2023Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experimentcited by 36
  9. 2025Scaling Embedding Layers in Language Modelscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026From Token to Token Pair: Efficient Prompt Compression for Large Language Models in Clinical Predictioncited by 2
  2. 2026UniCompress: Token Compression for Unified Vision-Language Understanding and Generationcited by 1
  3. 2026Lossless Prompt Compression via Dictionary-Encoding and In-Context Learning: Enabling Cost-Effective LLM Analysis of Repetitive Datacited by 0
  4. 2026Compressing Sequences in the Latent Embedding Space: K-Token Merging for Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Lossless Compression of Large Language Model-Generated Text via Next-Token Prediction
  2. 2026Lexical Prompt Compression for Large Language Models: A Training-Free, Deterministic Pipeline with Empirical Pareto Analysis Across Eleven Task Categories
  3. 2025Understanding and Improving Information Preservation in Prompt Compression for LLMs
  4. 2023A Survey on Model Compression for Large Language Models
  5. 2025Cmprsr: Abstractive Token-Level Question-Agnostic Prompt Compressor
  6. 2025Semantic Retention and Extreme Compression in LLMs: Can We Have Both?
  7. 2024Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models
  8. 2023Compressing LLMs: The Truth is Rarely Pure and Never Simple