mlsanityData to 7 Oct 2026

Privacy Guard & Token Parsimony by Prompt and Context Handling and LLM Routing

arXiv 2603.28972 ↗First posted 30 Mar 2026cs.CR cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

The large-scale adoption of Large Language Models (LLMs) forces a trade-off between operational cost (OpEx) and data privacy. Current routing frameworks reduce costs but ignore prompt sensitivity, exposing users and institutions to leakage risks towards third-party cloud providers. We formalise the "Inseparability Paradigm": advanced context management intrinsically coincides with privacy management. We propose a local "Privacy Guard" -- a holistic contextual observer powered by an on-premise Small Language Model (SLM) -- that performs abstractive summarisation and Automatic Prompt Optimisation (APO) to decompose prompts into focused sub-tasks, re-routing high-risk queries to Zero-Trust or NDA-covered models. This dual mechanism simultaneously eliminates sensitive inference vectors (Zero Leakage) and reduces cloud token payloads (OpEx Reduction). A LIFO-based context compacting mechanism further bounds working memory, limiting the emergent leakage surface. We validate the framework through a 2x2 benchmark (Lazy vs. Expert users; Personal vs. Institutional secrets) on a 1,000-sample dataset, achieving a 45% blended OpEx reduction, 100% redaction success on personal secrets, and -- via LLM-as-a-Judge evaluation -- an 85% preference rate for APO-compressed responses over raw baselines. Our results demonstrate that Token Parsimony and Zero Leakage are mathematically dual projections of the same contextual compression operator.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

20 of 27 references.

References · 20

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  3. 2023A Survey of Large Language Modelscited by 2,537
  4. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  5. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  7. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  8. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  9. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  10. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  11. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
  12. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
8 more
  1. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
  2. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  3. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 167
  4. 2023GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformercited by 84
  5. 2024Membership Inference Attacks Against In-Context Learningcited by 38
  6. 2024Prompt Leakage effect and defense strategies for multi-turn LLM interactionscited by 23
  7. 2024Reconstruct Your Previous Conversations! Comprehensively Investigating Privacy Leakage Risks in Conversations with GPT Modelscited by 16
  8. 2025CompactPrompt: A Unified Pipeline for Prompt Data Compression in LLM Workflowscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.