mlsanityData to 7 Oct 2026

Privacy-preserved LLM Cascade via CoT-enhanced Policy Learning

arXiv 2410.08014 ↗First posted 10 Oct 2024cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have gained significant attention in on-device applications due to their remarkable performance across real-world tasks. However, on-device LLMs often suffer from suboptimal performance due to hardware limitations. A promising solution to this challenge is cascading a weaker local (on-device) LLM with a more powerful server LLM. While existing research on LLM cascade primarily optimizes the performance-cost trade-off, real-world applications impose additional requirements, such as privacy preservation, which remain largely unaddressed. In this work, we move beyond existing confidence- and logit-based LLM cascade methods and propose mathbfP3Defer, a novel Chain-of-Thought (CoT)-enhanced policy learning framework for privacy-preserved deferral decision-making. Our approach effectively improves cascade efficiency while mitigating privacy risks. Extensive experiments on three benchmark datasets demonstrate the effectiveness and superiority of mathbfP3Defer over existing methods.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 35 of its 60 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

35 of 60 references.

References · 35

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  7. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  8. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
  9. 2023Large Language Model Routing with Benchmark Datasetscited by 166
  10. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 140
  11. 2024Towards Safer Large Language Models through Machine Unlearningcited by 134
  12. 2024Language Model Cascades: Token-level uncertainty and beyondcited by 115
23 more
  1. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 115
  2. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  3. 2023Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Methodcited by 97
  4. 2023Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modelingcited by 94
  5. 2018Offline Multi-Action Policy Learning: Generalization and Optimizationcited by 64
  6. 2023When Does Confidence-Based Cascade Deferral Suffice?cited by 58
  7. 2023LLM-based Medical Assistant Personalization with Short- and Long-Term Memory Coordinationcited by 46
  8. 2023LLMCad: Fast and Scalable On-device Large Language Model Inferencecited by 45
  9. 2024LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generationcited by 45
  10. 2024Towards Optimizing the Costs of LLM Usagecited by 33
  11. 2024Personalized LLM Response Generation with Parameterized Memory Injectioncited by 33
  12. 2023Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuningcited by 26
  13. 2021EmailSum: Abstractive Email Thread Summarizationcited by 24
  14. 2024Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignmentcited by 24
  15. 2024Learning to Refuse: Towards Mitigating Privacy Risks in LLMscited by 22
  16. 2024Cascade-Aware Training of Language Modelscited by 15
  17. 2024No Free Lunch Theorem for Privacy-Preserving LLM Inferencecited by 13
  18. 2024Efficient Contextual LLM Cascades through Budget-Constrained Policy Learningcited by 11
  19. 2023Ask more, know better: Reinforce-Learned Prompt Questions for Decision Making with Large Language Modelscited by 9
  20. 2024Can LLMs get help from other LLMs without revealing private information?cited by 9
  21. 2023Towards an On-device Agent for Text Rewritingcited by 7
  22. 2024Get Confused Cautiously: Textual Sequence Memorization Erasure with Selective Entropy Maximizationcited by 4
  23. 2019Rethinking Classification and Localization for Cascade R-CNNcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

  1. 2026PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Trainingcited by 6
  2. 2025Harnessing Chain-of-Thought Metadata for Task Routing and Adversarial Prompt Detectioncited by 5
  3. 2025A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthinesscited by 5
  4. 2025Economic Evaluation of LLMscited by 4
  5. 2025Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitationcited by 3
  6. 2026Is Escalation Worth It? On the Depth of LLM Cascadescited by 3
  7. 2026SEAL-Tag: Self-Tag Evidence Aggregation with Probabilistic Circuits for PII-Safe Retrieval-Augmented Generationcited by 0
  8. 2026When Cloud Agents Meet Device Agents: Lessons from Hybrid Multi-Agent Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LLMs as Linguistic Chameleons: Decoupling Semantics and Structure for Privacy-Preserving Communication
  2. 2026Securing LLMs in the Wild: Privacy and Security Challenges at the Edge
  3. 2025AVEC: Bootstrapping Privacy for Local LLMs
  4. 2026Operationalising the Right to be Forgotten in LLMs: A Lightweight Sequential Unlearning Framework for Privacy-Aligned Deployment in Politically Sensitive Environments
  5. 2025Towards Confidential and Efficient LLM Inference with Dual Privacy Protection
  6. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory
  7. 2026P2Skill: Privacy Preserving Skill Distillation for Cloud-Local LLM Inference Systems
  8. 2026Safer Reasoning Traces: Measuring and Mitigating Chain-of-Thought Leakage in LLMs