mlsanityData to 14 Oct 2026

Real Money, Fake Models: Deceptive Model Claims in Shadow APIs

arXiv 2603.01919 ↗First posted 2 Mar 2026cs.CR cs.AI cs.SEcited by 8

Not on the map. Why?

Abstract (arXiv)

Access to frontier large language models (LLMs), such as GPT-5 and Gemini-2.5, is often hindered by high pricing, payment barriers, and regional restrictions. These limitations drive the proliferation of textitshadow APIs, third-party services that claim to provide access to official model services without regional limitations via indirect access. Despite their widespread use, it remains unclear whether shadow APIs deliver outputs consistent with those of the official APIs, raising concerns about the reliability of downstream applications and the validity of research findings that depend on them. In this paper, we present the first systematic audit between official LLM APIs and corresponding shadow APIs. We first identify 17 shadow APIs that have been utilized in 187 academic papers, with the most popular one reaching more than 5,900 citations and 58,000 GitHub stars by December 6, 2025. Through multidimensional auditing of three representative shadow APIs across utility, safety, and model verification, we uncover widespread behavioral inconsistency and fingerprint-based evidence consistent with deceptive model claims in a subset of audited endpoints. Specifically, we reveal performance divergence reaching up to 47.21%, significant unpredictability in safety behaviors, and identity verification failures in 45.83% of fingerprint tests. These practices critically undermine the reproducibility and validity of scientific research, harm the interests of shadow API users, and damage the reputation of official model providers. By the time of writing, 4 of the 17 providers have already ceased operations, underscoring the operational volatility of this market. Meanwhile, unverifiable compliance claims and independent model-substitution testing platforms have emerged in the ecosystem, reflecting growing community awareness of this risk.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 41 of its 77 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

41 of 77 references.

References · 41

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  9. 2023A Survey of Large Language Modelscited by 2,576
  10. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  11. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  12. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
29 more
  1. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  2. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  3. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  4. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  5. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  6. 2024A StrongREJECT for Empty Jailbreakscited by 385
  7. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 336
  8. 2023Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvaturecited by 287
  9. 2019Thieves on Sesame Street! Model Extraction of BERT-based APIscited by 139
  10. 2023DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Textcited by 129
  11. 2025Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understandingcited by 102
  12. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 80
  13. 2024LLMmap: Fingerprinting For Large Language Modelscited by 58
  14. 2024Model Equality Testing: Which Model Is This API Serving?cited by 47
  15. 2025Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluationcited by 42
  16. 2025MedicalAgentsBench for Complex Medical Reasoning: Comparing Internalized Reasoning Models versus Externalized Agent-based Frameworkscited by 40
  17. 2025Are You Getting What You Pay For? Auditing Model Substitution in LLM APIscited by 38
  18. 2024Your Large Language Models Are Leaving Fingerprintscited by 33
  19. 2016Understanding the Factors that Impact the Popularity of GitHub Repositoriescited by 27
  20. 2026"Humans welcome to observe": A First Look at the Agent Social Network Moltbookcited by 26
  21. 2024textttModSCAN: Measuring Stereotypical Bias in Large Vision-Language Models from Vision and Language Modalitiescited by 22
  22. 2025Adjacent Words, Divergent Intents: Jailbreaking Large Language Models via Task Concurrencycited by 17
  23. 2025Domaino1s: Guiding LLM Reasoning for Explainable Answers in High-Stakes Domainscited by 11
  24. 2025Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyondcited by 9
  25. 2026Sparse Models, Sparse Safety: Unsafe Routes in Mixture-of-Experts LLMscited by 9
  26. 2024SVIP: Towards Verifiable Inference of Open-source Large Language Modelscited by 6
  27. 2025Analyzing 16,193 LLM Papers for Fun and Profitscited by 4
  28. 2025Unveiling the Landscape of LLM Deployment in the Wild: An Empirical Studycited by 4
  29. 2025Advanced Black-Box Tuning of Large Language Models with Limited API Callscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClawcited by 20
  2. 2026HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?cited by 11
  3. 2026AEX: Non-Intrusive Multi-Hop Attestation and Provenance for LLM APIscited by 3
  4. 2026KBF: Knowledge Boundary as Fingerprint for Language Model and Black-Box API Auditingcited by 3
  5. 2026Your "Pro" LLM Subscription May Actually Be "Free": Exposing Fingerprint Spoofing Risks in LLM Inference Servicescited by 3
  6. 2026Where Is the Tradeoff in Using Third-Party API Routers for Agentic Software Development?cited by 0
  7. 2026AgentProv: Auditing Agentic LLM API Providers via Tool-use Policy Probescited by 0
  8. 2026How Reproducible Are Evaluation Conclusions? A Self-Audit of LLM-Inferred Prompt Structurecited by 0