mlsanityData to 7 Oct 2026

Automatically Auditing Large Language Models via Discrete Optimization

arXiv 2303.04381 ↗First posted 8 Mar 2023cs.LG cs.CLcited by 187

Not on the map. Why?

Cited by 120 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Auditing large language models for unexpected behaviors is critical to preempt catastrophic deployments, yet remains challenging. In this work, we cast auditing as an optimization problem, where we automatically search for input-output pairs that match a desired target behavior. For example, we might aim to find a non-toxic input that starts with "Barack Obama" that a model maps to a toxic output. This optimization problem is difficult to solve as the set of feasible points is sparse, the space is discrete, and the language models we audit are non-linear and high-dimensional. To combat these challenges, we introduce a discrete optimization algorithm, ARCA, that jointly and efficiently optimizes over inputs and outputs. Our approach automatically uncovers derogatory completions about celebrities (e.g. "Barack Obama is a legalized unborn" -> "child murderer"), produces French inputs that complete to English outputs, and finds inputs that generate a specific name. Our work offers a promising new tool to uncover models' failure-modes before deployment.

Key facts. Cited by 187 papers in the map (data through 7 Oct 2026). Builds on 51 of its 123 references that are in the map. First posted on arXiv 8 Mar 2023.

Builds on

51 of 123 references.

References · 51

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  9. 2020Learning to summarize from human feedbackcited by 2,562
  10. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  11. 2020Extracting Training Data from Large Language Modelscited by 2,144
  12. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
38 more
  1. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  2. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  3. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  4. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  5. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  6. 2022Red Teaming Language Models with Language Modelscited by 983
  7. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  8. 2021Ethical and social risks of harm from Language Modelscited by 957
  9. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  10. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  11. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  12. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  13. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  14. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  15. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  16. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  17. 2018Generating Natural Language Adversarial Examplescited by 422
  18. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  19. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
  20. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  21. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  22. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  23. 2016FastText.zip: Compressing text classification modelscited by 331
  24. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  25. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  26. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  27. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 159
  28. 2018Counterfactual Fairness in Text Classification through Robustnesscited by 150
  29. 2021Detoxifying Language Models Risks Marginalizing Minority Voicescited by 106
  30. 2022Capturing Failures of Large Language Models via Human Cognitive Biasescited by 97
  31. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  32. 2021Controlled Text Generation as Continuous Optimization with Multiple Constraintscited by 70
  33. 2018Conversational AI: The Science Behind the Alexa Prizecited by 63
  34. 2022Adversarial Training for High-Stakes Reliabilitycited by 60
  35. 2020Investigating African-American Vernacular English in Transformer-Based Text Generationcited by 52
  36. 2022A Langevin-like Sampler for Discrete Distributionscited by 42
  37. 2018Unrestricted Adversarial Examplescited by 40
  38. 2021Solving Probability and Statistics Problems by Program Synthesiscited by 9

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 28; 2024: 99; 2025: 47; 2026: 13.

187 papers.

Sort:

By area:

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  3. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  5. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  6. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  7. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  8. 2023Detecting Language Model Attacks with Perplexitycited by 411
  9. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  10. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  11. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
  12. 2023Are aligned neural networks adversarially aligned?cited by 305
38 more
  1. 2023Low-Resource Languages Jailbreak GPT-4cited by 290
  2. 2023Large Language Model Alignment: A Surveycited by 241
  3. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  4. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  5. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  6. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  7. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  8. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  9. 2023Model evaluation for extreme riskscited by 167
  10. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  11. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  12. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  13. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  14. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 124
  15. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  16. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 123
  17. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120
  18. 2024Dissecting Adversarial Robustness of Multimodal LM Agentscited by 118
  19. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 111
  20. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Modelscited by 102
  21. 2024Coercing LLMs to do and reveal (almost) anythingcited by 88
  22. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 85
  23. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 70
  24. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  25. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  26. 2024CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Modelscited by 65
  27. 2024AdvAgent: Controllable Blackbox Red-teaming on Web Agentscited by 57
  28. 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 50
  29. 2024PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Railscited by 49
  30. 2024LatentQA: Teaching LLMs to Decode Activations Into Natural Languagecited by 49
  31. 2024PAL: Proxy-Guided Black-Box Attack on Large Language Modelscited by 46
  32. 2023Can LLMs Follow Simple Rules?cited by 45
  33. 2024Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Modelscited by 45
  34. 2024GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Modelscited by 44
  35. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  36. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Modelscited by 42
  37. 2024Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Ratescited by 42
  38. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42

50 most cited of 187 All 187 (JSON)