# AbuseGPT: Abuse of Generative AI ChatBots to Create Smishing Campaigns

arXiv: https://arxiv.org/abs/2402.09728 · HTML: https://mlsanity.com/paper/2402.09728/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study](https://mlsanity.com/paper/2305.13860/) (2023)
- [MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots](https://mlsanity.com/paper/2307.08715/) (2023)
- [From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy](https://mlsanity.com/paper/2307.00691/) (2023)
- [Exploring the Dark Side of AI: Advanced Phishing Attack Design and Deployment Using ChatGPT](https://mlsanity.com/paper/2309.10463/) (2023)
- [Users really do respond to smishing](https://mlsanity.com/paper/2212.13312/) (2022)
- [Characterizing the Landscape of COVID-19 Themed Cyberattacks and Defenses](https://mlsanity.com/paper/2102.13223/) (2021)
- [Data-Driven Characterization and Detection of COVID-19 Themed Malicious Websites](https://mlsanity.com/paper/2102.13226/) (2021)

## Built on by (most cited first, 9 of 9)

- [On the (In)Security of LLM App Stores](https://mlsanity.com/paper/2407.08422/) (2024)
- [Opportunities and Challenges of Generative-AI in Finance](https://mlsanity.com/paper/2410.15653/) (2024)
- [EVOREFUSE: Evolutionary Prompt Optimization for Evaluation and Mitigation of LLM Over-Refusal to Pseudo-Malicious Instructions](https://mlsanity.com/paper/2505.23473/) (2025)
- [TRIDENT: Enhancing Large Language Model Safety with Tri-Dimensional Diversified Red-Teaming Data Synthesis](https://mlsanity.com/paper/2505.24672/) (2025)
- [Send to which account? Evaluation of an LLM-based Scambaiting System](https://mlsanity.com/paper/2509.08493/) (2025)
- [Application of AI-based Models for Online Fraud Detection and Analysis](https://mlsanity.com/paper/2409.19022/) (2024)
- [Can Features for Phishing URL Detection Be Trusted Across Diverse Datasets? A Case Study with Explainable AI](https://mlsanity.com/paper/2411.09813/) (2024)
- [Jailbreaking Generative AI: Empowering Novices to Conduct Phishing Attacks](https://mlsanity.com/paper/2503.01395/) (2025)
- [Jailbreaking Generative AI: Multivector Phishing Threats and Transformer based Defenses](https://mlsanity.com/paper/2507.12185/) (2025)

## Abstract (arXiv)

SMS phishing, also known as "smishing", is a growing threat that tricks users into disclosing private information or clicking into URLs with malicious content through fraudulent mobile text messages. In recent past, we have also observed a rapid advancement of conversational generative AI chatbot services (e.g., OpenAI's ChatGPT, Google's BARD), which are powered by pre-trained large language models (LLMs). These AI chatbots certainly have a lot of utilities but it is not systematically understood how they can play a role in creating threats and attacks. In this paper, we propose AbuseGPT method to show how the existing generative AI-based chatbot services can be exploited by attackers in real world to create smishing texts and eventually lead to craftier smishing campaigns. To the best of our knowledge, there is no pre-existing work that evidently shows the impacts of these generative text-based models on creating SMS phishing. Thus, we believe this study is the first of its kind to shed light on this emerging cybersecurity threat. We have found strong empirical evidences to show that attackers can exploit ethical standards in the existing generative AI-based chatbot services by crafting prompt injection attacks to create newer smishing campaigns. We also discuss some future research directions and guidelines to protect the abuse of generative AI-based services and safeguard users from smishing attacks.