# A First Look at GPT Apps: Landscape and Vulnerability

arXiv: https://arxiv.org/abs/2402.15105 · HTML: https://mlsanity.com/paper/2402.15105/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 75 references that are in the map.
- First posted on arXiv 23 Feb 2024.

## Builds on (most cited first, 12 of 12)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- [Yara Parser: A Fast and Accurate Dependency Parser](https://mlsanity.com/paper/1503.06733/) (2015)
- [Assessing Prompt Injection Risks in 200+ Custom GPTs](https://mlsanity.com/paper/2311.11538/) (2023)
- [Cross-lingual Editing in Multilingual Language Models](https://mlsanity.com/paper/2401.10521/) (2024)
- [Opening A Pandora's Box: Things You Should Know in the Era of Custom GPTs](https://mlsanity.com/paper/2401.00905/) (2023)
- [AndroZoo++: Collecting Millions of Android Apps and Their Metadata for the Research Community](https://mlsanity.com/paper/1709.05281/) (2017)
- [Demystifying Removed Apps in iOS App Store](https://mlsanity.com/paper/2101.05100/) (2021)
- [A Longitudinal Study of Google Play](https://mlsanity.com/paper/1802.02996/) (2018)
- [Structural Analysis of User Choices for Mobile App Recommendation](https://mlsanity.com/paper/1605.07980/) (2016)

## Built on by (most cited first, 16 of 16)

- [What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use](https://mlsanity.com/paper/2409.08775/) (2024)
- [The Pitfalls of KV Cache Compression](https://mlsanity.com/paper/2510.00231/) (2025)
- [On the (In)Security of LLM App Stores](https://mlsanity.com/paper/2407.08422/) (2024)
- [An In-Depth Investigation of Data Collection in LLM App Ecosystems](https://mlsanity.com/paper/2408.13247/) (2024)
- [Do LLMs Know to Respect Copyright Notice?](https://mlsanity.com/paper/2411.01136/) (2024)
- [LLM Applications: Current Paradigms and the Next Frontier](https://mlsanity.com/paper/2503.04596/) (2025)
- [Unraveling Misinformation Propagation in LLM Reasoning](https://mlsanity.com/paper/2505.18555/) (2025)
- [GPT Store Mining and Analysis](https://mlsanity.com/paper/2405.10210/) (2024)
- [LLM App Squatting and Cloning](https://mlsanity.com/paper/2411.07518/) (2024)
- [Towards Safer Chatbots: Automated Policy Compliance Evaluation of Custom GPTs](https://mlsanity.com/paper/2502.01436/) (2025)
- [A Large-Scale Empirical Analysis of Custom GPTs' Vulnerabilities in the OpenAI Ecosystem](https://mlsanity.com/paper/2505.08148/) (2025)
- [SSRLBot: Designing and Developing a Large Language Model-based Agent using Socially Shared Regulated Learning](https://mlsanity.com/paper/2505.00945/) (2025)
- [When GPT Spills the Tea: Comprehensive Assessment of Knowledge File Leakage in GPTs](https://mlsanity.com/paper/2506.00197/) (2025)
- [LaQual: An Automated Framework for LLM App Quality Evaluation](https://mlsanity.com/paper/2508.18636/) (2025)
- [Beyond Jailbreak: Unveiling Risks in LLM Applications Arising from Blurred Capability Boundaries](https://mlsanity.com/paper/2511.17874/) (2025)
- [Do No Harm? Hallucination and Actor-Level Abuse in Web-Deployed Medical Large Language Models](https://mlsanity.com/paper/2605.20591/) (2026)

## Abstract (arXiv)

Following OpenAI's introduction of GPTs, a surge in GPT apps has led to the launch of dedicated LLM app stores. Nevertheless, given its debut, there is a lack of sufficient understanding of this new ecosystem. To fill this gap, this paper presents a first comprehensive longitudinal (5-month) study of the evolution, landscape, and vulnerability of the emerging LLM app ecosystem, focusing on two GPT app stores: textitGPTStore.AI and the official textitOpenAI GPT Store. Specifically, we develop two automated tools and a TriLevel configuration extraction strategy to efficiently gather metadata (ie names, creators, descriptions, etc) and user feedback for all GPT apps across these two stores, as well as configurations (ie system prompts, knowledge files, and APIs) for the top 10,000 popular apps. Our extensive analysis reveals: (1) the user enthusiasm for GPT apps consistently rises, whereas creator interest plateaus within three months of GPTs' launch; (2) nearly 90% system prompts can be easily accessed due to widespread failure to secure GPT app configurations, leading to considerable plagiarism and duplication among apps. Our findings highlight the necessity of enhancing the LLM app ecosystem by the app stores, creators, and users.