mlsanityData to 14 Oct 2026

OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Languages and Modalities

arXiv 2505.23856 ↗First posted 29 May 2025cs.CL cs.AI cs.HCcited by 7

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

The emerging capabilities of large language models (LLMs) have sparked concerns about their immediate potential for harmful misuse. The core approach to mitigate these concerns is the detection of harmful queries to the model. Current detection approaches are fallible, and are particularly susceptible to attacks that exploit mismatched generalization of model capabilities (e.g., prompts in low-resource languages or prompts provided in non-text modalities such as image and audio). To tackle this challenge, we propose Omniguard, an approach for detecting harmful prompts across languages and modalities. Our approach (i) identifies internal representations of an LLM/MLLM that are aligned across languages or modalities and then (ii) uses them to build a language-agnostic or modality-agnostic classifier for detecting harmful prompts. Omniguard improves harmful prompt classification accuracy by 11.57% over the strongest baseline in a multilingual setting, by 20.44% for image-based prompts, and sets a new SOTA for audio-based prompts. By repurposing embeddings computed during generation, Omniguard is also very efficient (≈!120 × faster than the next fastest baseline). Code and data are available at: https://github.com/vsahil/OmniGuard.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 54 of its 74 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

54 of 74 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Enhancing LLM Safety Through a Theoretical Minimax Game Lens7 Feb
  2. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails15 Jan
  3. 2024Benchmarking LLM Guardrails in Handling Multilingual Toxicity29 Oct
  4. 2024Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture10 Jul
  5. 2024The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm26 Jun
  6. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs26 Jun
  7. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completion12 Mar

Other references · 47

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  7. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  8. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  9. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  10. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  11. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  12. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
35 more
  1. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  2. 2023Detecting Language Model Attacks with Perplexitycited by 431
  3. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  4. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
  5. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  6. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 398
  7. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  8. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  9. 2023Low-Resource Languages Jailbreak GPT-4cited by 309
  10. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  11. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
  12. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 230
  13. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
  14. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
  15. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 191
  16. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 151
  17. 2024How do Large Language Models Handle Multilingualism?cited by 150
  18. 2024Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversationscited by 129
  19. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 103
  20. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 69
  21. 2024The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalitiescited by 69
  22. 2024Jailbreak Large Vision-Language Models Through Multi-Modal Linkagecited by 62
  23. 2025PolyGuard: A Multilingual Safety Moderation Tool for 17 Languagescited by 59
  24. 2024MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilitiescited by 57
  25. 2024Semantic Mirror Jailbreak: Genetic Algorithm Based Jailbreak Prompts Against Open-source LLMscited by 53
  26. 2024When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Searchcited by 47
  27. 2024Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characterscited by 42
  28. 2022Augmenting Interpretable Models with LLMs during Trainingcited by 38
  29. 2023Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysiscited by 35
  30. 2023Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Informationcited by 33
  31. 2024Sandwich attack: Multi-language Mixture Adaptive Attack on LLMscited by 20
  32. 2024VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wildcited by 18
  33. 2024Vector-ICL: In-context Learning with Continuous Vector Representationscited by 13
  34. 2024Crafting Interpretable Embeddings by Asking LLMs Questionscited by 11
  35. 2024Toxicity Classification in Ukrainiancited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025Qwen3Guard Technical Reportcited by 155
  2. 2025CultureGuard: Towards Culturally-Aware Dataset and Guard Model for Multilingual Safety Applicationscited by 27
  3. 2025DynaGuard: A Dynamic Guardian Model With User-Defined Policiescited by 19
  4. 2025Kelp: A Streaming Safeguard for Large Models via Latent Dynamics-Guided Risk Detectioncited by 9
  5. 2026Omni-Safety under Cross-Modality Conflict: Vulnerabilities, Dynamics Mechanisms and Efficient Alignmentcited by 6
  6. 2026AEGIS: Audio Endogenous Guarding via Internal Signals Against Large Audio-Language Model Jailbreakscited by 1
  7. 2026Benchmarking Jailbreak Guardrails for Embodied Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SGuard-v1: Safety Guardrail for Large Language Models
  2. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
  3. 2026Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models
  4. 2024Large Language Model Safety: A Holistic Survey
  5. 2024On Prompt-Driven Safeguarding for Large Language Models
  6. 2026Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguards
  7. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models
  8. 2025OmniGuard: Unified Omni-Modal Guardrails with Deliberate Reasoning