# ToViLaG: Your Visual-Language Generative Model is Also An Evildoer

arXiv: https://arxiv.org/abs/2312.11523 · HTML: https://mlsanity.com/paper/2312.11523/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 84 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Builds on (most cited first, 30 of 62)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework](https://mlsanity.com/paper/2202.03052/) (2022)

## Built on by (most cited first, 27 of 27)

- [Safety of Multimodal Large Language Models on Images and Texts](https://mlsanity.com/paper/2402.00357/) (2024)
- [Align Anything: Training All-Modality Models to Follow Instructions with Language Feedback](https://mlsanity.com/paper/2412.15838/) (2024)
- [Assessment of Multimodal Large Language Models in Alignment with Human Values](https://mlsanity.com/paper/2403.17830/) (2024)
- [LLMs Meet Multimodal Generation and Editing: A Survey](https://mlsanity.com/paper/2405.19334/) (2024)
- [Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks](https://mlsanity.com/paper/2502.13175/) (2025)
- [From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking](https://mlsanity.com/paper/2406.14859/) (2024)
- [Adversarial Attacks of Vision Tasks in the Past 10 Years: A Survey](https://mlsanity.com/paper/2410.23687/) (2024)
- [SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIP](https://mlsanity.com/paper/2408.10202/) (2024)
- [Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models](https://mlsanity.com/paper/2410.12662/) (2024)
- [Automating Steering for Safe Multimodal Large Language Models](https://mlsanity.com/paper/2507.13255/) (2025)
- [MoHoBench: Assessing Honesty of Multimodal Large Language Models via Unanswerable Visual Questions](https://mlsanity.com/paper/2507.21503/) (2025)
- [CHAI: Command Hijacking against embodied AI](https://mlsanity.com/paper/2510.00181/) (2025)
- [Zero-Shot Defense Against Toxic Images via Inherent Multimodal Alignment in LVLMs](https://mlsanity.com/paper/2503.00037/) (2025)
- [REVAL: A Comprehension Evaluation on Reliability and Values of Large Vision-Language Models](https://mlsanity.com/paper/2503.16566/) (2025)
- [REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLM](https://mlsanity.com/paper/2505.04673/) (2025)
- [Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation](https://mlsanity.com/paper/2509.07596/) (2025)
- [Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigation](https://mlsanity.com/paper/2508.15370/) (2025)
- [Embedding an Ethical Mind: Aligning Text-to-Image Synthesis via Lightweight Value Optimization](https://mlsanity.com/paper/2410.12700/) (2024)
- [How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigation](https://mlsanity.com/paper/2502.14486/) (2025)
- [Q-MLLM: Vector Quantization for Robust Multimodal Large Language Model Security](https://mlsanity.com/paper/2511.16229/) (2025)
- [ExtremeAIGC: Benchmarking LMM Vulnerability to AI-Generated Extremist Content](https://mlsanity.com/paper/2503.09964/) (2025)
- [MDIT-Bench: Evaluating the Dual-Implicit Toxicity in Large Multimodal Models](https://mlsanity.com/paper/2505.17144/) (2025)
- [VModA: An Effective Framework for Adaptive NSFW Image Moderation](https://mlsanity.com/paper/2505.23386/) (2025)
- [Old Tricks, New Models: How Simple Image Transformations Break Modern AI-based Content Moderation](https://mlsanity.com/paper/2607.28187/) (2026)
- [Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Models](https://mlsanity.com/paper/2608.29590/) (2026)
- [SPARK: Representation-Level KV Memory Alignment for Safer Vision-Language Models](https://mlsanity.com/paper/2609.14258/) (2026)
- [ShieldCLIP: Selective Safety Alignment for Harmful Content Mitigation in Multimodal Foundation Models](https://mlsanity.com/paper/2609.39688/) (2026)

## Abstract (arXiv)

Warning: this paper includes model outputs showing offensive content. Recent large-scale Visual-Language Generative Models (VLGMs) have achieved unprecedented improvement in multimodal image/text generation. However, these models might also generate toxic content, e.g., offensive text and pornography images, raising significant ethical risks. Despite exhaustive studies on toxic degeneration of language models, this problem remains largely unexplored within the context of visual-language generation. This work delves into the propensity for toxicity generation and susceptibility to toxic data across various VLGMs. For this purpose, we built ToViLaG, a dataset comprising 32K co-toxic/mono-toxic text-image pairs and 1K innocuous but evocative text that tends to stimulate toxicity. Furthermore, we propose WInToRe, a novel toxicity metric tailored to visual-language generation, which theoretically reflects different aspects of toxicity considering both input and output. On such a basis, we benchmarked the toxicity of a diverse spectrum of VLGMs and discovered that some models do more evil than expected while some are more vulnerable to infection, underscoring the necessity of VLGMs detoxification. Therefore, we develop an innovative bottleneck-based detoxification method. Our method could reduce toxicity while maintaining comparable generation quality, providing a promising initial solution to this line of research.