mlsanityData to 7 Oct 2026

Computational Safety for Generative AI: A Hypothesis Testing Perspective

arXiv 2502.12445 ↗First posted 18 Feb 2025cs.AI cs.LG stat.MLcited by 0

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

AI safety is a rapidly growing area of research that seeks to prevent the harm and misuse of frontier AI technology, particularly with respect to generative AI (GenAI) tools that are capable of creating realistic and high-quality content through text prompts. Examples of such tools include large language models (LLMs) and text-to-image (T2I) diffusion models. As the performance of various leading GenAI models approaches saturation due to similar training data sources and neural network architecture designs, the development of reliable safety guardrails has become a key differentiator for responsibility and sustainability. This paper presents a formalization of the concept of computational safety, which is a mathematical framework that enables the quantitative assessment, formulation, and study of safety challenges in GenAI through the lens of signal processing theory and methods. In particular, we explore two exemplary categories of computational safety challenges in GenAI that can be formulated as hypothesis testing problems. For the safety of model input, we show how sensitivity analysis and loss landscape analysis can be used to detect malicious prompts with jailbreak attempts. For the safety of model output, we elucidate how statistical signal processing can be used to detect AI-generated content. Finally, we discuss key open research challenges, opportunities, and the essential role of signal processing in computational AI safety.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 34 of its 41 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

34 of 41 references.

References · 34

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  10. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  11. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  12. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
22 more
  1. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 705
  2. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  3. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  4. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  5. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  6. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  7. 2020Media Forensics and DeepFakes: an overviewcited by 208
  8. 2023RADAR: Robust AI-Text Detection via Adversarial Learningcited by 187
  9. 2020A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learningcited by 160
  10. 2024DF40: Toward Next-Generation Deepfake Detectioncited by 126
  11. 2024AEROBLADE: Training-Free Detection of Latent Diffusion Images Using Autoencoder Reconstruction Errorcited by 123
  12. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
  13. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  14. 2022Model Reprogramming: Resource-Efficient Cross-Domain Machine Learningcited by 64
  15. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapescited by 63
  16. 2024RIGID: A Training-free and Model-Agnostic Framework for Robust AI-Generated Image Detectioncited by 36
  17. 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attackscited by 35
  18. 2024AI Safety in Generative AI Large Language Models: A Surveycited by 31
  19. 2017Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recoverycited by 25
  20. 2024AI Risk Management Should Incorporate Both Safety and Securitycited by 17
  21. 2024Defining and Evaluating Physical Safety for Large Language Modelscited by 9
  22. 2024Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Modelscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Identifying and Mitigating the Security Risks of Generative AI
  2. 2024A Formal Framework for Assessing and Mitigating Emergent Security Risks in Generative AI Models: Bridging Theory and Dynamic Risk Mitigation
  3. 2025GenBreak: Red Teaming Text-to-Image Generators Using Large Language Models
  4. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety
  5. 2024Concept-Guided LLM Agents for Human-AI Safety Codesign
  6. 2024Large Language Model Safety: A Holistic Survey
  7. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Experts
  8. 2026Yuvion LLM: An Adversarially-Aware Large Language Model for Content And AI Safety