mlsanityData to 7 Oct 2026

Beyond the Safeguards: Exploring the Security Risks of ChatGPT

arXiv 2305.08005 ↗First posted 13 May 2023cs.CR cs.AI cs.CLcited by 36

Not on the map. Why?

Abstract (arXiv)

The increasing popularity of large language models (LLMs) such as ChatGPT has led to growing concerns about their safety, security risks, and ethical implications. This paper aims to provide an overview of the different types of security risks associated with ChatGPT, including malicious text and code generation, private data disclosure, fraudulent services, information gathering, and producing unethical content. We present an empirical study examining the effectiveness of ChatGPT's content filters and explore potential ways to bypass these safeguards, demonstrating the ethical implications and security risks that persist in LLMs even when protections are in place. Based on a qualitative analysis of the security implications, we discuss potential strategies to mitigate these risks and inform researchers, policymakers, and industry professionals about the complex security challenges posed by LLMs like ChatGPT. This study contributes to the ongoing discussion on the ethical and security implications of LLMs, underscoring the need for continued research in this area.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 28 of its 40 references that are in the map. First posted on arXiv 13 May 2023.

Builds on

28 of 40 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2017Deep reinforcement learning from human preferencescited by 3,846
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2020Extracting Training Data from Large Language Modelscited by 2,144
  9. 2022Red Teaming Language Models with Language Modelscited by 983
  10. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  11. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  12. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 306
16 more
  1. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  2. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  3. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  4. 2021Understanding the Capabilities, Limitations, and Societal Impact of Large Language Modelscited by 159
  5. 2021Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Modelscited by 141
  6. 2021Detoxifying Language Models Risks Marginalizing Minority Voicescited by 106
  7. 2023Regulating ChatGPT and other Large Generative AI Modelscited by 102
  8. 2023BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPTcited by 70
  9. 2019Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?cited by 68
  10. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  11. 2021Mitigating harm in language models with conditional-likelihood filtrationcited by 38
  12. 2021Generating Fake Cyber Threat Intelligence Using Transformer-Based Modelscited by 28
  13. 2022Targeted Phishing Campaigns using Large Scale Language Modelscited by 22
  14. 2022Adaptive Sampling Strategies to Construct Equitable Training Datasetscited by 18
  15. 2023Perspectives on the Social Impacts of Reinforcement Learning with Human Feedbackcited by 15
  16. 2023Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvementscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 18; 2025: 5; 2026: 2.

36 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  2. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  3. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  4. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstructioncited by 95
  5. 2023GPTEval: A Survey on Assessments of ChatGPT and GPT-4cited by 73
  6. 2024A Complete Survey on LLM-based AI Chatbotscited by 71
  7. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 65
  8. 2024AI Safety in Generative AI Large Language Models: A Surveycited by 31
  9. 2023Distilled GPT for Source Code Summarizationcited by 30
  10. 2024The Ethics of Interaction: Mitigating Security Threats in LLMscited by 30
  11. 2024Farsight: Fostering Responsible AI Awareness During AI Application Prototypingcited by 28
  12. 2023On the Detectability of ChatGPT Content: Benchmarking, Methodology, and Evaluation through the Lens of Academic Writingcited by 26
24 more
  1. 2023Using Large Language Models for Cybersecurity Capture-The-Flag Challenges and Certification Questionscited by 23
  2. 2024Towards Trustworthy AI: A Review of Ethical and Robust Large Language Modelscited by 21
  3. 2024Mapping the Ethics of Generative AI: A Comprehensive Scoping Reviewcited by 19
  4. 2023RatGPT: Turning online LLMs into Proxies for Malware Attackscited by 17
  5. 2023Efficient Avoidance of Vulnerabilities in Auto-completed Smart Contract Code Using Vulnerability-constrained Decodingcited by 14
  6. 2023From Chatbots to PhishBots? -- Preventing Phishing scams created using ChatGPT, Google Bard and Claudecited by 12
  7. 2024SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobilecited by 12
  8. 2024Risk and Response in Large Language Models: Evaluating Key Threat Categoriescited by 10
  9. 2023Ask Again, Then Fail: Large Language Models' Vacillations in Judgmentcited by 8
  10. 2024Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiencycited by 7
  11. 2023Learning to Prompt in the Classroom to Understand AI Limits: A pilot studycited by 6
  12. 2024FreezeAsGuard: Mitigating Illegal Adaptation of Diffusion Models via Selective Tensor Freezingcited by 4
  13. 2024Unanswerability Evaluation for Retrieval Augmented Generationcited by 4
  14. 2024Tagengo: A Multilingual Chat Datasetcited by 3
  15. 2024Whispers in the Machine: Confidentiality in Agentic Systemscited by 2
  16. 2025SOK: Exploring Hallucinations and Security Risks in AI-Assisted Software Development with Insights for LLM Deploymentcited by 2
  17. 2025Securing Educational LLMs: A Generalised Taxonomy of Attacks on LLMs and DREAD Risk Assessmentcited by 2
  18. 2024Is On-Device AI Broken and Exploitable? Assessing the Trust and Ethics in Small Language Modelscited by 1
  19. 2025Two Birds with One Stone: Multi-Task Detection and Attribution of LLM-Generated Textcited by 1
  20. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  21. 2026LanG -- A Governance-Aware Agentic AI Platform for Unified Security Operationscited by 1
  22. 2026Reasoning and Planning with Dynamically Changing Normscited by 1
  23. 2024Improving Dialog Safety using Socially Aware Contrastive Learningcited by 0
  24. 2025AI Ethics and Social Norms: Exploring ChatGPT's Capabilities From What to Howcited by 0