mlsanityData to 14 Oct 2026

PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles

arXiv 2410.17127 ↗First posted 22 Oct 2024cs.CR cs.CLcited by 57

Not on the map. Why?

Cited by 21 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Users can divulge sensitive information to proprietary LLM providers, raising significant privacy concerns. While open-source models, hosted locally on the user's machine, alleviate some concerns, models that users can host locally are often less capable than proprietary frontier models. Toward preserving user privacy while retaining the best quality, we propose Privacy-Conscious Delegation, a novel task for chaining API-based and local models. We utilize recent public collections of user-LLM interactions to construct a natural benchmark called PUPA, which contains personally identifiable information (PII). To study potential approaches, we devise PAPILLON, a multi-stage LLM pipeline that uses prompt optimization to address a simpler version of our task. Our best pipeline maintains high response quality for 85.5% of user queries while restricting privacy leakage to only 7.5%. We still leave a large margin to the generation quality of proprietary LLMs for future work. Our data and code is available at https://github.com/siyan-sylvia-li/PAPILLON.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Builds on 24 of its 52 references that are in the map. First posted on arXiv 22 Oct 2024.

Builds on

24 of 52 references.

References · 24

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  7. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  8. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  9. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 291
  10. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
  11. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
  12. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
12 more
  1. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 186
  2. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
  3. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 119
  4. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 89
  5. 2020Training Production Language Models without Memorizing User Datacited by 68
  6. 2023DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineercited by 57
  7. 2020A Differentially Private Text Perturbation Method Using a Regularized Mahalanobis Metriccited by 42
  8. 2023Joint Prompt Optimization of Stacked LLMs using Variational Inferencecited by 32
  9. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 28
  10. 2024ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?cited by 17
  11. 2024An In-Depth Investigation of Data Collection in LLM App Ecosystemscited by 14
  12. 2023ChatGPT as a Therapist Assistant: A Suitability Studycited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 24; 2026: 33.

57 papers.

Sort:

By area:

  1. 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakagecited by 69
  2. 2025Feedback Descent: Open-Ended Text Optimization via Pairwise Comparisoncited by 44
  3. 2025Towards Trustworthy GUI Agents: A Surveycited by 34
  4. 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkerscited by 28
  5. 2025Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agentscited by 26
  6. 2025Searching for Privacy Risks in LLM Agents via Simulationcited by 22
  7. 2025Minions: Cost-efficient Collaboration Between On-device and Cloud Language Modelscited by 20
  8. 2025Position: Privacy Is Not Just Memorization!cited by 17
  9. 2025How Robust Are Router-LLMs? Analysis of the Fragility of LLM Routing Capabilitiescited by 11
  10. 2025SALT: Steering Activations towards Leakage-free Thinking in Chain of Thoughtcited by 11
  11. 2025Operationalizing Data Minimization for Privacy-Preserving LLM Promptingcited by 8
  12. 2025The Surprising Effectiveness of Membership Inference with Simple N-Gram Coveragecited by 7
38 more
  1. 2025VortexPIA: Indirect Prompt Injection Attack against LLMs for Efficient Extraction of User Privacycited by 7
  2. 2026Learning Next Action Predictors from Human-Computer Interactioncited by 7
  3. 2025Automated Privacy Information Annotation in Large Language Model Interactionscited by 6
  4. 2025PPMI: Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databasescited by 6
  5. 2025Privacy-R1: Privacy-Aware Multi-LLM Agent Collaboration via Reinforcement Learningcited by 6
  6. 2026LEVI: Stronger Search Architectures Can Substitute for Larger LLMs in Evolutionary Searchcited by 6
  7. 2025Can Large Language Models Really Recognize Your Name?cited by 4
  8. 2025Composing Policy Gradients and Prompt Optimization for Language Model Programscited by 4
  9. 2026POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agentscited by 4
  10. 2025AutoSpec: An Agentic Framework for Automatically Drafting Patent Specificationcited by 3
  11. 2026Privasis: Synthesizing the Largest "Public" Private Dataset from Scratchcited by 3
  12. 2026AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systemscited by 3
  13. 2025Learning Obfuscations Of LLM Embedding Sequences: Stained Glass Transformcited by 2
  14. 2026AlienLM: Alienization of Language for API-Boundary Privacy in Black-Box LLMscited by 2
  15. 2026Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)cited by 2
  16. 2026MAC: Multi-Agent Constitution Learningcited by 2
  17. 2026Local-First Intelligence via Proactive SLM Collaborating with Feedback-Providing LLMcited by 2
  18. 2026Minim: Privacy-Aware Minimal View for Agents via Trusted Local Sanitizationcited by 2
  19. 2026PrivacyAlign: Contextual Privacy Alignment for LLM Agentscited by 2
  20. 2026Beyond Direct Identifiers: Probabilistic Privacy Risk Estimation for Privacy-Conscious LLM Query Delegationcited by 2
  21. 2025ASTRA: An Automated Framework for Strategy Discovery, Retrieval, and Evolution for Jailbreaking LLMscited by 1
  22. 2026Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimizationcited by 1
  23. 2026MaskClaw: Edge-Side Personalized Privacy Arbitration for GUI Agents with Behavior-Driven Skill Evolutioncited by 1
  24. 2026FAPO: Fully Automated Prompt Optimization of Multi-Step LLM Pipelinescited by 1
  25. 2026ToolMinimize: Auditing and Rewriting LLM Agent Tool Calls to Minimize Privacy Exposurecited by 1
  26. 2026Demystifying the Privacy-Utility Trade-off in LLM Interactionscited by 1
  27. 2026Selection-Invariant Communication Compilers for Privacy-Aware Multi-Agent LLM Workflowscited by 1
  28. 2025Collaborative LLM Numerical Reasoning with Local Data Protectioncited by 0
  29. 2025Stereo-LiDAR Fusion by Semi-Global Matching With Discrete Disparity-Matching Cost and Semidensificationcited by 0
  30. 2025Adaptive Backtracking for Privacy Protection in Large Language Modelscited by 0
  31. 2026BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelinescited by 0
  32. 2026Automated Instruction Revision (AIR): A Structured Comparison of Task Adaptation Strategies for LLMcited by 0
  33. 2026Assessing Privacy Preservation and Utility in Online Vision-Language Modelscited by 0
  34. 2026FlowBot: Inducing LLM Workflows with Bilevel Optimization and Textual Gradientscited by 0
  35. 2026Prompt Codebooks: Discrete Compositional Optimization for Language Model Instruction Refinementcited by 0
  36. 2026LLM Anonymization Against Agentic Re-Identificationcited by 0
  37. 2026ToFu: A White-Box, Token-Efficient Agent Harness for Researcherscited by 0
  38. 2026Reward-Free Evolving Agents via Pairwise Validatorcited by 0

50 most cited of 57 All 57 (JSON)