mlsanityData to 7 Oct 2026

A LLM Assisted Exploitation of AI-Guardian

arXiv 2307.15008 ↗First posted 20 Jul 2023cs.CR cs.AI cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are now highly capable at a diverse range of tasks. This paper studies whether or not GPT-4, one such LLM, is capable of assisting researchers in the field of adversarial machine learning. As a case study, we evaluate the robustness of AI-Guardian, a recent defense to adversarial examples published at IEEE S&P 2023, a top computer security conference. We completely break this defense: the proposed scheme does not increase robustness compared to an undefended baseline. We write none of the code to attack this model, and instead prompt GPT-4 to implement all attack algorithms following our instructions and guidance. This process was surprisingly effective and efficient, with the language model at times producing code from ambiguous instructions faster than the author of this paper could have done. We conclude by discussing (1) the warning signs present in the evaluation that suggested to us AI-Guardian would be broken, and (2) our experience with designing attacks and performing novel research using the most recent advances in language modeling.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 24 of its 44 references that are in the map. First posted on arXiv 20 Jul 2023.

Builds on

24 of 44 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  8. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  9. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  10. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  11. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  12. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
12 more
  1. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  2. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  3. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
  4. 2019On Evaluating Adversarial Robustnesscited by 442
  5. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  6. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  7. 2017MagNet: a Two-Pronged Defense against Adversarial Examplescited by 268
  8. 2017The Space of Transferable Adversarial Examplescited by 156
  9. 2019The Odds are Odd: A Statistical Test for Detecting Adversarial Examplescited by 75
  10. 2019Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networkscited by 40
  11. 2021Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descentcited by 34
  12. 2020A Partial Break of the Honeypots Defense to Catch Adversarial Attackscited by 3

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 7; 2025: 4.

14 papers.

Sort:

By area:

  1. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  2. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  3. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  4. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 51
  5. 2024RevPRAG: Revealing Poisoning Attacks in Retrieval-Augmented Generation through LLM Activation Analysiscited by 33
  6. 2025AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defensescited by 14
  7. 2024Constrained Adaptive Attack: Effective Adversarial Attack Against Deep Neural Networks for Tabular Datacited by 11
  8. 2024TabularBench: Benchmarking Adversarial Robustness for Tabular Deep Learning in Real-world Use-casescited by 10
  9. 2025Automated Profile Inference with Language Model Agentscited by 9
  10. 2023Vulnerability Analysis of Transformer-based Optical Character Recognition to Adversarial Attackscited by 2
  11. 2024Cutting through buggy adversarial example defenses: fixing 1 line of code breaks Sabrecited by 2
  12. 2025Evaluating the efficacy of LLM Safety Solutions : The Palit Benchmark Datasetcited by 2
2 more
  1. 2023Can We Rely on AI?cited by 0
  2. 2024Interactive Simulations of Backdoors in Neural Networkscited by 0