mlsanityData to 7 Oct 2026

RogueGPT: dis-ethical tuning transforms ChatGPT4 into a Rogue AI in 158 Words

arXiv 2407.15009 ↗First posted 11 Jun 2024cs.CY cs.AI cs.CLcited by 4

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

The ethical implications and potentials for misuse of Generative Artificial Intelligence are increasingly worrying topics. This paper explores how easily the default ethical guardrails of ChatGPT, using its latest customization features, can be bypassed by simple prompts and fine-tuning, that can be effortlessly accessed by the broad public. This malevolently altered version of ChatGPT, nicknamed "RogueGPT", responded with worrying behaviours, beyond those triggered by jailbreak prompts. We conduct an empirical study of RogueGPT responses, assessing its flexibility in answering questions pertaining to what should be disallowed usage. Our findings raise significant concerns about the model's knowledge about topics like illegal drug production, torture methods and terrorism. The ease of driving ChatGPT astray, coupled with its global accessibility, highlights severe issues regarding the data quality used for training the foundational model and the implementation of ethical safeguards. We thus underline the responsibilities and dangers of user-driven modifications, and the broader effects that these may have on the design of safeguarding and ethical modules implemented by AI programmers.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 24 of its 49 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

24 of 49 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  9. 2018Model Cards for Model Reportingcited by 1,068
  10. 2016Generative Adversarial Text to Image Synthesiscited by 735
  11. 2020Aligning AI With Shared Human Valuescited by 706
  12. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
12 more
  1. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  2. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  3. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
  4. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  5. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  6. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  7. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  8. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  9. 2019A Comprehensive Study on Deep Learning Bug Characteristicscited by 65
  10. 2024EasyJailbreak: A Unified Framework for Jailbreaking Large Language Modelscited by 57
  11. 2024ChatGPT vs Gemini vs LLaMA on Multilingual Sentiment Analysiscited by 13
  12. 2023A Comparative Study of Code Generation using ChatGPT 3.5 across 10 Programming Languagescited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025FAIRGAME: a Framework for AI Agents Bias Recognition using Game Theorycited by 13
  2. 2025Do LLMs trust AI regulation? Emerging behaviour of game-theoretic LLM agentscited by 11
  3. 2025Mind the Language Gap: Automated and Augmented Evaluation of Bias in LLMs for High- and Low-Resource Languagescited by 4
  4. 2026Assessing Rule Adherence of LLM Adjudicators in Call of Cthulhu TRPGcited by 0