# Attacks on Third-Party APIs of Large Language Models

arXiv: https://arxiv.org/abs/2404.16891 · HTML: https://mlsanity.com/paper/2404.16891/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 24 Apr 2024.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- [API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs](https://mlsanity.com/paper/2304.08244/) (2023)
- [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023)
- [BERT-ATTACK: Adversarial Attack Against BERT Using BERT](https://mlsanity.com/paper/2004.09984/) (2020)
- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [An LLM can Fool Itself: A Prompt-Based Adversarial Attack](https://mlsanity.com/paper/2310.13345/) (2023)
- [From Prompt Injections to SQL Injection Attacks: How Protected is Your LLM-Integrated Web Application?](https://mlsanity.com/paper/2308.01990/) (2023)
- [LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins](https://mlsanity.com/paper/2309.10254/) (2023)
- [LLMDet: A Third Party Large Language Models Generated Text Detection Tool](https://mlsanity.com/paper/2305.15004/) (2023)

## Built on by (most cited first, 15 of 15)

- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools](https://mlsanity.com/paper/2508.02110/) (2025)
- [Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystem](https://mlsanity.com/paper/2506.02040/) (2025)
- [From Storage to Steering: Memory Control Flow Attacks on LLM Agents](https://mlsanity.com/paper/2603.15125/) (2026)
- [Analysing the Residual Stream of Language Models Under Knowledge Conflicts](https://mlsanity.com/paper/2410.16090/) (2024)
- [Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Risk](https://mlsanity.com/paper/2502.15865/) (2025)
- [DAWN: Designing Distributed Agents in a Worldwide Network](https://mlsanity.com/paper/2410.22339/) (2024)
- [Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implications](https://mlsanity.com/paper/2509.08604/) (2025)
- [Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods](https://mlsanity.com/paper/2605.05227/) (2026)
- [When Agents Overtrust Environmental Evidence: An Extensible Agentic Framework for Benchmarking Evidence-Grounding Defects in LLM Agents](https://mlsanity.com/paper/2605.08828/) (2026)
- [A Large-Scale Empirical Analysis of Custom GPTs' Vulnerabilities in the OpenAI Ecosystem](https://mlsanity.com/paper/2505.08148/) (2025)
- [StackingNet: Collective Inference Across Independent AI Foundation Models](https://mlsanity.com/paper/2602.13792/) (2026)
- [Learning to Solve Complex Problems via Dataset Decomposition](https://mlsanity.com/paper/2602.20296/) (2026)
- [Security Attack and Defense Strategies for Autonomous Agent Frameworks: A Layered Review with OpenClaw as a Case Study](https://mlsanity.com/paper/2604.27464/) (2026)
- [Beware EviLLM: Enabling Vulnerability Injection via Large Language Models](https://mlsanity.com/paper/2610.03857/) (2026)

## Abstract (arXiv)

Large language model (LLM) services have recently begun offering a plugin ecosystem to interact with third-party API services. This innovation enhances the capabilities of LLMs, but it also introduces risks, as these plugins developed by various third parties cannot be easily trusted. This paper proposes a new attacking framework to examine security and safety vulnerabilities within LLM platforms that incorporate third-party services. Applying our framework specifically to widely used LLMs, we identify real-world malicious attacks across various domains on third-party APIs that can imperceptibly modify LLM outputs. The paper discusses the unique challenges posed by third-party API integration and offers strategic possibilities to improve the security and safety of LLM ecosystems moving forward. Our code is released at https://github.com/vk0812/Third-Party-Attacks-on-LLMs.