# An In-Depth Investigation of Data Collection in LLM App Ecosystems

arXiv: https://arxiv.org/abs/2408.13247 · HTML: https://mlsanity.com/paper/2408.13247/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 65 references that are in the map.
- First posted on arXiv 23 Aug 2024.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Long-context LLMs Struggle with Long In-context Learning](https://mlsanity.com/paper/2404.02060/) (2024)
- [Analyzing Leakage of Personally Identifiable Information in Language Models](https://mlsanity.com/paper/2302.00539/) (2023)
- [Beyond Memorization: Violating Privacy Via Inference with Large Language Models](https://mlsanity.com/paper/2310.07298/) (2023)
- [LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins](https://mlsanity.com/paper/2309.10254/) (2023)
- [Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learning](https://mlsanity.com/paper/1802.02561/) (2018)
- [A First Look at GPT Apps: Landscape and Vulnerability](https://mlsanity.com/paper/2402.15105/) (2024)
- [Do Cookie Banners Respect my Choice? Measuring Legal Compliance of Banners from IAB Europe's Transparency and Consent Framework](https://mlsanity.com/paper/1911.09964/) (2019)
- [Cookie Synchronization: Everything You Always Wanted to Know But Were Afraid to Ask](https://mlsanity.com/paper/1805.10505/) (2018)
- [OVRseen: Auditing Network Traffic and Privacy Policies in Oculus VR](https://mlsanity.com/paper/2106.05407/) (2021)
- [Opted Out, Yet Tracked: Are Regulations Enough to Protect Your Privacy?](https://mlsanity.com/paper/2202.00885/) (2022)
- [PURL: Safe and Effective Sanitization of Link Decoration](https://mlsanity.com/paper/2308.03417/) (2023)

## Built on by (most cited first, 14 of 14)

- [PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles](https://mlsanity.com/paper/2410.17127/) (2024)
- [LLMs for Explainable AI: A Comprehensive Survey](https://mlsanity.com/paper/2504.00125/) (2025)
- [Towards Automating Data Access Permissions in AI Agents](https://mlsanity.com/paper/2511.17959/) (2025)
- [HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?](https://mlsanity.com/paper/2604.15415/) (2026)
- [GLiNER2: An Efficient Multi-Task Information Extraction System with Schema-Driven Interface](https://mlsanity.com/paper/2507.18546/) (2025)
- [Behavioral Integrity Verification for AI Agent Skills](https://mlsanity.com/paper/2605.11770/) (2026)
- [AI-VERDE: A Gateway for Egalitarian Access to Large Language Model-Based Resources For Educational Institutions](https://mlsanity.com/paper/2502.09651/) (2025)
- [No Attack Required: Semantic Fuzzing for Specification Violations in Agent Skills](https://mlsanity.com/paper/2605.13044/) (2026)
- [A Longitudinal Measurement of Privacy Policy Evolution for Large Language Models](https://mlsanity.com/paper/2511.21758/) (2025)
- [OET: Optimization-based prompt injection Evaluation Toolkit](https://mlsanity.com/paper/2505.00843/) (2025)
- [Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact](https://mlsanity.com/paper/2605.14021/) (2026)
- [LaQual: An Automated Framework for LLM App Quality Evaluation](https://mlsanity.com/paper/2508.18636/) (2025)
- [Behavioral Canaries: Auditing Private Retrieved Context Usage in RL Fine-Tuning](https://mlsanity.com/paper/2604.22191/) (2026)
- [PromptPET: Privacy-Utility Optimized Prompt Obfuscation](https://mlsanity.com/paper/2607.02932/) (2026)

## Abstract (arXiv)

LLM app (tool) ecosystems are rapidly evolving to support sophisticated use cases that often require extensive user data collection. Given that LLM apps are developed by third parties and anecdotal evidence indicating inconsistent enforcement of policies by LLM platforms, sharing user data with these apps presents significant privacy risks. In this paper, we aim to bring transparency in data practices of LLM app ecosystems. We examine OpenAI's GPT app ecosystem as a case study. We propose an LLM-based framework to analyze the natural language specifications of GPT Actions (custom tools) and assess their data collection practices. Our analysis reveals that Actions collect excessive data across 24 categories and 145 data types, with third-party Actions collecting 6.03% more data on average. We find that several Actions violate OpenAI's policies by collecting sensitive information, such as passwords, which is explicitly prohibited by OpenAI. Lastly, we develop an LLM-based privacy policy analysis framework to automatically check the consistency of data collection by Actions with disclosures in their privacy policies. Our measurements indicate that the disclosures for most of the collected data types are omitted, with only 5.8% of Actions clearly disclosing their data collection practices.