mlsanityData to 7 Oct 2026

Mass-Producing Failures of Multimodal Systems with Language Models

arXiv 2306.12105 ↗First posted 21 Jun 2023cs.LG cs.CL cs.SEcited by 36

Not on the map. Why?

Abstract (arXiv)

Deployed multimodal systems can fail in ways that evaluators did not anticipate. In order to find these failures before deployment, we introduce MultiMon, a system that automatically identifies systematic failures -- generalizable, natural-language descriptions of patterns of model failures. To uncover systematic failures, MultiMon scrapes a corpus for examples of erroneous agreement: inputs that produce the same output, but should not. It then prompts a language model (e.g., GPT-4) to find systematic patterns of failure and describe them in natural language. We use MultiMon to find 14 systematic failures (e.g., "ignores quantifiers") of the CLIP text-encoder, each comprising hundreds of distinct inputs (e.g., "a shelf with a few/many books"). Because CLIP is the backbone for most state-of-the-art multimodal systems, these inputs produce failures in Midjourney 5.1, DALL-E, VideoFusion, and others. MultiMon can also steer towards failures relevant to specific use cases, such as self-driving cars. We see MultiMon as a step towards evaluation that autonomously explores the long tail of potential system failures. Code for MULTIMON is available at https://github.com/tsb0601/MultiMon.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 55 of its 124 references that are in the map. First posted on arXiv 21 Jun 2023.

Builds on

55 of 124 references.

References · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  10. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  11. 2020Extracting Training Data from Large Language Modelscited by 2,144
  12. 2015A large annotated corpus for learning natural language inferencecited by 1,837
38 more
  1. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  3. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  4. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  5. 2022Red Teaming Language Models with Language Modelscited by 983
  6. 2021Ethical and social risks of harm from Language Modelscited by 957
  7. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  8. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  9. 2023Extracting Training Data from Diffusion Modelscited by 723
  10. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
  11. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  12. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  13. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  14. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  15. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  16. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  17. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  18. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  19. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  20. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  21. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  22. 2022Red-Teaming the Stable Diffusion Safety Filtercited by 257
  23. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  24. 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 157
  25. 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 155
  26. 2013Stabilitycited by 119
  27. 2021Algorithmic Monoculture and Social Welfarecited by 98
  28. 2022Capturing Failures of Large Language Models via Human Cognitive Biasescited by 97
  29. 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 89
  30. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  31. 2019Bias Correction of Learned Generative Models using Likelihood-Free Importance Weightingcited by 70
  32. 2023Goal Driven Discovery of Distributional Differences via Language Descriptionscited by 61
  33. 2022Testing Relational Understanding in Text-Guided Image Generationcited by 59
  34. 2023Diagnosing and Rectifying Vision Models using Languagecited by 58
  35. 2022Describing Differences between Text Distributions with Natural Languagecited by 54
  36. 2018Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failurecited by 49
  37. 2022Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioningcited by 45
  38. 2020Adversarial Semantic Collisionscited by 42

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 16; 2025: 8; 2026: 6.

36 papers.

Sort:

By area:

  1. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  2. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  3. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  4. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
  5. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  6. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 103
  7. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  8. 2024Visual Hallucinations of Multi-modal Large Language Modelscited by 68
  9. 2023TokenCompose: Text-to-Image Diffusion with Token-level Supervisioncited by 40
  10. 2024Understanding and Improving Training-free Loss-based Diffusion Guidancecited by 31
  11. 2024Adversaries Can Misuse Combinations of Safe Modelscited by 31
  12. 2024What Factors Affect Multi-Modal In-Context Learning? An In-Depth Explorationcited by 31
24 more
  1. 2025CLIP Behaves like a Bag-of-Words Model Cross-modally but not Uni-modallycited by 30
  2. 2024Finetuned Multimodal Language Models Are High-Quality Image-Text Data Filterscited by 25
  3. 2024MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation modelscited by 22
  4. 2023Effective Human-AI Teams via Learned Natural Language Rules and Onboardingcited by 20
  5. 2024A Philosophical Introduction to Language Models - Part II: The Way Forwardcited by 17
  6. 2024Anytime Continual Learning for Open Vocabulary Classificationcited by 8
  7. 2024Evaluating Vision-Language Models as Evaluators in Path Planningcited by 6
  8. 2023ASPIRE: Language-Guided Data Augmentation for Improving Robustness Against Spurious Correlationscited by 5
  9. 2025What's Producible May Not Be Reachable: Measuring the Steerability of Generative Modelscited by 5
  10. 2023Identifying Systematic Errors in Object Detectors with the SCROD Pipelinecited by 4
  11. 2025Uncovering Gaps in How Humans and LLMs Interpret Subjective Languagecited by 4
  12. 2025GHOST: Hallucination-Inducing Image Generation for Multimodal LLMscited by 4
  13. 2024Sound Check: Auditing Audio Datasetscited by 3
  14. 2024Unveiling Glitches: A Deep Dive into Image Encoding Bugs within CLIPcited by 1
  15. 2025Discovering Divergent Representations between Text-to-Image Modelscited by 1
  16. 2025Uncovering the Computational Ingredients of Human-Like Representations in LLMscited by 1
  17. 2026All-in-One Conditioning for Text-to-Image Synthesiscited by 1
  18. 2026Multi-Modal Learning meets Genetic Programming: Analyzing Alignment in Latent Space Optimizationcited by 1
  19. 2026When Do Diffusion Models learn to Generate Multiple Objects?cited by 1
  20. 2024Identification of Fine-grained Systematic Errors via Controlled Scene Generationcited by 0
  21. 2025What's in Common? Multimodal Models Hallucinate When Reasoning Across Scenescited by 0
  22. 2026Make it SING: Analyzing Semantic Invariants in Classifierscited by 0
  23. 2026How can embedding models bind concepts?cited by 0
  24. 2026DriftScope: Measuring The Hidden Effects of Diffusion Model Adaptationcited by 0