mlsanityData to 7 Oct 2026

Failures Are Fated, But Can Be Faded: Characterizing and Mitigating Unwanted Behaviors in Large-Scale Vision and Language Models

arXiv 2406.07145 ↗First posted 11 Jun 2024cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

In large deep neural networks that seem to perform surprisingly well on many tasks, we also observe a few failures related to accuracy, social biases, and alignment with human values, among others. Therefore, before deploying these models, it is crucial to characterize this failure landscape for engineers to debug and legislative bodies to audit models. Nevertheless, it is infeasible to exhaustively test for all possible combinations of factors that could lead to a model's failure. In this paper, we introduce a post-hoc method that utilizes deep reinforcement learning to explore and construct the landscape of failure modes in pre-trained discriminative and generative models. With the aid of limited human feedback, we then demonstrate how to restructure the failure landscape to be more desirable by moving away from the discovered failure modes. We empirically show the effectiveness of the proposed method across common Computer Vision, Natural Language Processing, and Vision-Language tasks.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 30 of its 37 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

30 of 37 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2017Deep reinforcement learning from human preferencescited by 3,846
  7. 2020Learning to summarize from human feedbackcited by 2,562
  8. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  9. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  10. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  11. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  12. 2023Preference Ranking Optimization for Human Alignmentcited by 283
18 more
  1. 2014One weird trick for parallelizing convolutional neural networkscited by 264
  2. 2016Safety Verification of Deep Neural Networkscited by 262
  3. 2021Deep learning: a statistical viewpointcited by 239
  4. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  5. 2017Deep Reinforcement Learning for De-Novo Drug Designcited by 161
  6. 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 155
  7. 2020Chip Placement with Deep Reinforcement Learningcited by 83
  8. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  9. 2020A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systemscited by 71
  10. 2020Opportunities and Challenges in Deep Learning Adversarial Robustness: A Surveycited by 71
  11. 2020PatchAttack: A Black-box Texture-based Attack with Reinforcement Learningcited by 67
  12. 2023LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Imagescited by 46
  13. 2020Out-of-Distribution Detection for Automotive Perceptioncited by 37
  14. 2022Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learningcited by 27
  15. 2022How Do We Fail? Stress Testing Perception in Autonomous Vehiclescited by 17
  16. 2021Reinforcement Learning Based Sparse Black-box Adversarial Attack on Video Recognition Modelscited by 12
  17. 2024The Role of Predictive Uncertainty and Diversity in Embodied AI and Robot Learningcited by 11
  18. 2021Evidential Softmax for Sparse Multimodal Distributions in Deep Generative Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 2; 2026: 2.

11 papers.

  1. 2024Leveraging Environment Interaction for Automated PDDL Translation and Planning with Large Language Modelscited by 17
  2. 2025When Continue Learning Meets Multimodal Large Language Model: A Surveycited by 8
  3. 2024RoboMD: Uncovering Robot Vulnerabilities through Semantic Potential Fieldscited by 5
  4. 2025Uncertainty in Action: Confidence Elicitation in Embodied Agentscited by 5
  5. 2024Explainable Concept Generation through Vision-Language Preference Learning for Understanding Neural Networks' Internal Representationscited by 3
  6. 2026Generating Data-Driven Reasoning Rubrics for Domain-Adaptive Reward Modelingcited by 1
  7. 2024Towards Adapting Reinforcement Learning Agents to New Tasks: Insights from Q-Valuescited by 0
  8. 2024BaTCAVe: Trustworthy Explanations for Robot Behaviorscited by 0
  9. 2024Constructive Apraxia: An Unexpected Limit of Instructible Vision-Language Models and Analog for Human Cognitive Disorderscited by 0
  10. 2024LLM-Assisted Red Teaming of Diffusion Models through "Failures Are Fated, But Can Be Faded"cited by 0
  11. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0