mlsanityData to 7 Oct 2026

Sensitive-Topic Leakage Through LLM Routing Metadata: Measurement and Mitigation

arXiv 2610.09981 ↗First posted 7 Oct 2026cs.CR cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

LLM routers pick a cheap or expensive model per request by its content, and many gateways and some cloud platforms can log that choice with content logging off. We measure this privacy channel beyond token counts, accounting for noisy labels and repeated prompts. We run pre-registered studies on 1.7 million real requests (WildChat-1M, LMSYS-Chat-1M) with two cost/quality routers and a domain router, survey eleven systems' logging, and test post-processing defenses. At matched length, the shift's direction depends on category and router. For RouteLLM at the 50% operating point, harassment and self-harm requests reach the strong model 19 points less often than comparable ones on prompts unseen in exploration, medical requests (exploratory: LLM labels failed their gate) 31 points less often on distinct prompts (both post hoc), and sexual requests 10 points more often (secondary); the other router's four are negative. Twenty RouteLLM decisions separate frequent medical askers with AUC 0.71, exploratory and below the pre-registered primary endpoint's 0.75 (domain router: 0.92, an upper estimate). Per-category length-matched parity with accurate labels removes the gap on real traffic, costing at most 0.2 accuracy points on RouterBench (post hoc), where routers' gaps on sensitive subjects (13-42 points, pre-registered) exceed those of an oracle routing by realized accuracy gain (1-11, post hoc). Per-conversation stickiness, per-user budget bands, and pooled parity fail, the last as categories' shifts differ in size or sign. A post hoc exact per-user rate hides only even-prefix strong counts and forfeits most self-assessed routing value; it preserves odd-position decisions, from which a post hoc log attack reaches AUC 0.73 after 20 RouteLLM requests (exploratory).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 42 references that are in the map. First posted on arXiv 7 Oct 2026.

Builds on

23 of 42 references.

References · 23

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  3. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  4. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  5. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  6. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  7. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  8. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  9. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
  10. 2023AutoMix: Automatically Mixing Language Modelscited by 116
  11. 2022Fair and Optimal Classification via Post-Processingcited by 50
  12. 2024Remote Timing Attacks on Efficient Language Model Inferencecited by 13
11 more
  1. 2025Whisper Leak: a side-channel attack on Large Language Modelscited by 3
  2. 2026RouteScan: A Non-Intrusive Approach to Auditing MoE LLMs Safety via Expert Routing Telemetrycited by 2
  3. 2026Task- and Session-Level Model Routing: A Common-Interface Hybrid Evaluation of Four Open-Source Routers Across Four Benchmarkscited by 2
  4. 2025AVEC: Bootstrapping Privacy for Local LLMscited by 1
  5. 2026Privacy-Preserving LLMs Routingcited by 1
  6. 2026From Privacy to Workflow Integrity: Communication-Graph Metadata in Autonomous Agent Interoperabilitycited by 1
  7. 2026Register Bias in Complexity-Based Large Language Model Routingcited by 1
  8. 2026Selection-Invariant Communication Compilers for Privacy-Aware Multi-Agent LLM Workflowscited by 1
  9. 2026A Large-Scale Benchmark and Risk Assessment of Traffic Analysis Attacks on Cloud LLM Servicescited by 1
  10. 2026Dynamic LLM Routers are Often Misguidedcited by 1
  11. 2026Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Usecited by 1

Built on by

Not cited yet.