mlsanityData to 7 Oct 2026

Towards the Worst-case Robustness of Large Language Models

arXiv 2501.19040 ↗First posted 31 Jan 2025cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Recent studies have revealed the vulnerability of large language models to adversarial attacks, where adversaries craft specific input sequences to induce harmful, violent, private, or incorrect outputs. In this work, we study their worst-case robustness, i.e., whether an adversarial example exists that leads to such undesirable outputs. We upper bound the worst-case robustness using stronger white-box attacks, indicating that most current deterministic defenses achieve nearly 0% worst-case robustness. We propose a general tight lower bound for randomized smoothing using fractional knapsack solvers or 0-1 knapsack solvers, and using them to bound the worst-case robustness of all stochastic defenses. Based on these solvers, we provide theoretical lower bounds for several previous empirical defenses. For example, we certify the robustness of a specific case, smoothing using a uniform kernel, against textitany possible attack with an average ℓ0 perturbation of 2.02 or an average suffix length of 6.41.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 100 of its 124 references that are in the map. First posted on arXiv 31 Jan 2025.

Builds on

100 of 124 references.

References · 100

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  11. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  12. 2013Intriguing properties of neural networkscited by 5,114
38 more
  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  6. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  7. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  9. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  10. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  12. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  13. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  14. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  15. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  16. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  17. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  18. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  19. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  20. 2022Diffusion Models for Adversarial Purificationcited by 469
  21. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  22. 2019On Evaluating Adversarial Robustnesscited by 442
  23. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  24. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  25. 2023Detecting Language Model Attacks with Perplexitycited by 411
  26. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  27. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  28. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  29. 2017Defense against Adversarial Attacks Using High-Level Representation Guided Denoisercited by 329
  30. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
  31. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
  32. 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 307
  33. 2023Large Language Models as Tool Makerscited by 293
  34. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  35. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  36. 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 254
  37. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  38. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 222

50 most cited of 100 All 100 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2026AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystemscited by 15
  2. 2025Unveiling the Basin-Like Loss Landscape in Large Language Modelscited by 13
  3. 2025Efficient Input-level Backdoor Defense on Text-to-Image Synthesis via Neuron Activation Variationcited by 8
  4. 2025Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrievalcited by 7
  5. 2025FABLE: A Localized, Targeted Adversarial Attack on Weather Forecasting Modelscited by 3
  6. 2025Intellectual Property in Graph-Based Machine Learning as a Service: Attacks and Defensescited by 2
  7. 2026Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothingcited by 0