The Violation State: Safety State Persistence in a Multimodal Language Model Interface
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
also in 1 more
Abstract (arXiv)
Multimodal AI systems integrate text generation, image generation, and other capabilities within a single conversational interface. These systems employ safety mechanisms to prevent disallowed actions, including the removal of watermarks from copyrighted images. While single-turn refusals are expected, the interaction between safety filters and conversation-level state is not well understood. This study documents a reproducible behavioral effect in the ChatGPT (GPT-5.1) web interface. Manual execution was chosen to capture the exact user-facing safety behavior of the production system, rather than isolated API components. When a conversation begins with an uploaded copyrighted image and a request to remove a watermark, which the model correctly refuses, subsequent prompts to generate unrelated, benign images are refused for the remainder of the session. Importantly, text-only requests (e.g., generating a Python function) continue to succeed. Across 40 manually run sessions (30 contaminated and 10 controls), contaminated threads showed 116/120 image-generation refusals (96.67%), while control threads showed 0/40 refusals (Fisher's exact p < 0.0001). All sessions used an identical fixed prompt order, ensuring sequence uniformity across conditions. We describe this as safety-state persistence: a form of conversational over-generalization in which a copyright refusal influences subsequent, unrelated image-generation behavior. We present these findings as behavioral observations, not architectural claims. We discuss possible explanations, methodological limitations (single model, single interface), and implications for multimodal reliability, user experience, and the design of session-level safety systems. These results motivate further examination of session-level safety interactions in multimodal AI systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 1 of its 4 references that are in the map. First posted on arXiv 18 Dec 2025.
Builds on
1 of 4 references.
References · 1
- 2022Training language models to follow instructions with human feedbackcited by 15,712
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CSR-Bench: A Benchmark for Evaluating the Cross-modal Safety and Reliability of MLLMs
- 2025SafeMT: Multi-turn Safety for Multimodal Language Models
- 2024Multimodal Situational Safety
- 2026MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialogues
- 2025Steering Multimodal Large Language Models Decoding for Context-Aware Safety
- 2025MSTS: A Multimodal Safety Test Suite for Vision-Language Models
- 2025LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Models
- 2025OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Models