Data to 7 Oct 2026

ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time

arXiv 2410.06625 ↗First posted 9 Oct 2024cs.CV cs.CL cs.LGcited by 57

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more

LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

Vision Language Models (VLMs) have become essential backbones for multimodal intelligence, yet significant safety challenges limit their real-world application. While textual inputs are often effectively safeguarded, adversarial visual inputs can easily bypass VLM defense mechanisms. Existing defense methods are either resource-intensive, requiring substantial data and compute, or fail to simultaneously ensure safety and usefulness in responses. To address these limitations, we propose a novel two-phase inference-time alignment framework, Evaluating Then Aligning (ETA): 1) Evaluating input visual contents and output responses to establish a robust safety awareness in multimodal settings, and 2) Aligning unsafe behaviors at both shallow and deep levels by conditioning the VLMs' generative distribution with an interference prefix and performing sentence-level best-of-N to search the most harmless and helpful generation paths. Extensive experiments show that ETA outperforms baseline methods in terms of harmlessness, helpfulness, and efficiency, reducing the unsafe rate by 87.5% in cross-modality attacks and achieving 96.6% win-ties in GPT-4 helpfulness evaluation. The code is publicly available at https://github.com/DripNowhy/ETA.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 52 of its 57 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

52 of 57 references.

Closest ancestors · 10

Same direction, newest first

Other references · 42

2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
2023GPT-4 Technical Reportcited by 18,132
2014Microsoft COCO: Common Objects in Contextcited by 17,498
2022Training language models to follow instructions with human feedbackcited by 15,712
2024The Llama 3 Herd of Modelscited by 14,192
2023Visual Instruction Tuningcited by 7,792
2023Improved Baselines with Visual Instruction Tuningcited by 4,362
2017Deep reinforcement learning from human preferencescited by 3,846
2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498

30 more

2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
2019Towards VQA Models That Can Readcited by 1,696
2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Expertscited by 369
2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
2023Self-Evaluation Guided Beam Search for Reasoningcited by 195
2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 103
2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 99
2024ARGS: Alignment as Reward-Guided Searchcited by 99
2023CLIPood: Generalizing CLIP to Out-of-Distributionscited by 96
2024Cascade Reward Sampling for Efficient Decoding-Time Alignmentcited by 38
2024Self-Evaluation as a Defense Against Adversarial Attacks on LLMscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 36; 2026: 19.

57 papers.

Sort:

By area:

2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 59
2025Safety in Large Reasoning Models: A Surveycited by 58
2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
2025Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injectioncited by 23
2025Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Modelscited by 21
2024The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defensecited by 20
2025Understanding and Rectifying Safety Perception Distortion in VLMscited by 17
2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
2025Analyzing Finetuning Representation Shift for Multimodal LLMs Steeringcited by 15
2025Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Modelscited by 13
2025SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimizationcited by 11

38 more

2025VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimizationcited by 10
2025SafePTR: Token-Level Jailbreak Defense in Multimodal LLMs via Prune-then-Restore Mechanismcited by 9
2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Modelscited by 8
2025DAVSP: Safety Alignment for Large Vision-Language Models via Deep Aligned Visual Safety Promptcited by 7
2025Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attackcited by 5
2025AMIA: Automatic Masking and Joint Intention Analysis Makes LVLMs Robust Jailbreak Defenderscited by 5
2025SHIELD: Classifier-Guided Prompting for Robust and Safer LVLMscited by 5
2025EcoAlign: An Economically Rational Framework for Efficient LVLM Alignmentcited by 5
2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5
2025Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalitiescited by 4
2025Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigationcited by 4
2025Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignmentcited by 4
2025Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gapcited by 3
2025The Safety Reminder: A Soft Prompt to Reactivate Delayed Safety Awareness in Vision-Language Modelscited by 3
2025SIA: Enhancing Safety via Intent Awareness for Vision-Language Modelscited by 3
2025Improving Alignment in LVLMs with Debiased Self-Judgmentcited by 3
2025SafeR-CLIP: Mitigating NSFW Content in Vision-Language Models While Preserving Pre-Trained Knowledgecited by 3
2026Understanding and Defending VLM Jailbreaks via Jailbreak-Related Representation Shiftcited by 3
2026WARD: Adversarially Robust Defense of Web Agents Against Prompt Injectionscited by 3
2025Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Modelscited by 2
2025A Survey on Training-free Alignment of Large Language Modelscited by 2
2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2
2025PRISM: Robust VLM Alignment with Principled Reasoning for Integrated Safety in Multimodalitycited by 2
2025Q-MLLM: Vector Quantization for Robust Multimodal Large Language Model Securitycited by 2
2026Principled Steering via Null-space Projection for Jailbreak Defense in Vision-Language Modelscited by 2
2026Visual Token Compression Enhances Robustness of MLLMscited by 2
2025Dynamic Token Reweighting for Robust Vision-Language Modelscited by 1
2025Aligning Large Vision-Language Models by Deep Reinforcement Learning and Direct Preference Optimizationcited by 1
2025VERA-V: Variational Inference Framework for Jailbreaking Vision-Language Modelscited by 1
2025Agentic Moderation: Multi-Agent Design for Safer Vision-Language Modelscited by 1
2026Robustness of Vision Language Models Against Split-Image Harmful Input Attackscited by 1
2026VALD: Multi-Stage Vision Attack Detection for Efficient LVLM Defensecited by 1
2026GuardAlign: Test-time Safety Alignment in Multimodal Large Language Modelscited by 1
2025Mitigating Object Hallucination via Robust Local Perception Searchcited by 0
2026Safety-Potential Pruning for Enhancing Safety Prompts Against VLM Jailbreaking Without Retrainingcited by 0
2026Two Birds, One Projection: Harmonizing Safety and Utility in LVLMs via Inference-time Feature Projectioncited by 0
2026MLingualFC: Evaluating Jailbreak Vulnerabilities in Multilingual Vision-Language Modelscited by 0
2026V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failurecited by 0

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way