Token Inflation: How Dishonest Providers Can Overcharge for Large Language Model Usage
Not on the map. Why?
Abstract (arXiv)
Per-token billing is now the standard pricing model for commercial large language models (LLMs), so the honesty of reported token counts directly affects what users pay. We show that this kind of billing is hard to audit by design: providers hide the model, the tokenizer, and the execution to protect their IP, mitigate jailbreaks, and preserve user privacy, which means an auditor can only inspect proofs the provider supplies. The audit therefore reduces to a consistency check on the provider's own reports. We call this a trust paradox: every audit must trust some artifact, but current frameworks trust exactly the ones a provider has the strongest reason to manipulate. We study three recent token auditing frameworks and show that a provider with ordinary commercial capabilities can systematically inflate billed token counts. In the most permissive setting, hidden reasoning usage can be inflated by 1,469% on average without detection. At current frontier reasoning prices, that turns a 100 honest bill into roughly a 1,569 bill on the same query. Even when the user can see the full reasoning string, tokenization ambiguity alone still allows 50.85% over-reporting below the detection threshold. These results suggest the problem is not in any specific auditor but in any audit whose evidence comes from the audited party. Restoring honest billing will require verification that ties reported token counts to evidence the provider does not control, such as trusted execution attestation, cryptographic proofs of inference, or third-party re-execution.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 28 May 2026.
Builds on
24 of 34 references.
References · 24
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
- 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
- 2024Stealing Part of a Production Language Modelcited by 159
- 2018Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardwarecited by 102
12 more
- 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinkingcited by 99
- 2024What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineeringcited by 90
- 2024Is Cosine-Similarity of Embeddings Really About Similarity?cited by 89
- 2025Are You Getting What You Pay For? Auditing Model Substitution in LLM APIscited by 38
- 2024The Foundations of Tokenization: Statistical and Computational Concernscited by 22
- 2025Auditing Black-Box LLM APIs with a Rank-Based Uniformity Testcited by 18
- 2025CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIscited by 14
- 2025Is Your LLM Overcharging You? Tokenization, Transparency, and Incentivescited by 13
- 2025Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Servicescited by 10
- 2025Predictive Auditing of Hidden Tokens in LLM APIs via Reasoning Length Estimationcited by 7
- 2024Can't Hide Behind the API: Stealing Black-Box Commercial Embedding Modelscited by 5
- 2025Auditing Pay-Per-Token in Large Language Modelscited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026IMMACULATE: A Practical LLM Auditing Framework via Verifiable Computation
- 2025TRUST: A Decentralized Framework for Auditing Large Language Model Reasoning
- 2026Learning the Cost of Reliable Inference
- 2026Thinking Costs Tokens: When More Structure is Worth the Price
- 2026Not All Tokens Are Equal: Inflation-Aware Routing for Agentic LLM Systems
- 2026Keep, Customize, or Exit: Default Design and Token Pricing in LLM Reasoning Services
- 2025Audit the Whisper: Detecting Steganographic Collusion in Multi-Agent LLMs
- 2026Utility-Aware Data Pricing: Token-Level Quality and Empirical Training Gain for LLMs