mlsanityData to 7 Oct 2026

QR-Structured Thermal Triggers for Targeted Semantic Attacks on Infrared Vision-Language Models

arXiv 2607.29445 ↗First posted 31 Jul 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Infrared vision-language models (IR-VLMs) extend thermal perception to open-vocabulary classification, image captioning, and visual question answering. However, their robustness to structured thermal perturbations and the stability of cross-modal semantic alignment remain insufficiently studied. We propose QR-Structured Thermal Triggers (QR-STT), a stealthy, training-free, black-box framework for targeted semantic steering of IR-VLMs. QR-STT preserves the functional regions of a QR pattern while optimizing its internal modules, each of which is assigned a cold, neutral, or hot thermal state. The framework jointly searches module topology and rendering parameters, including position, scale, rotation, intensity, blur, and roundness. A three-stage gradient-free procedure with greedy module-flip refinement efficiently handles the mixed discrete and continuous search space. The objective promotes alignment with an attacker-selected target, suppresses source-class evidence, and regularizes QR structure and visual similarity. Experiments on multiple CLIP-style encoders show that QR-STT consistently redirects image-text alignment toward chosen concepts while maintaining visual stealth. Perturbations optimized for classification also transfer to image captioning and VQA, causing target-consistent semantic drift in generated outputs. These results identify QR-structured thermal patterns as an interpretable attack surface for language-driven infrared perception and highlight the need for robustness evaluation against structured cross-task semantic attacks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 28 references that are in the map. First posted on arXiv 31 Jul 2026.

Builds on

14 of 28 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025V-Attack: Targeting Disentangled Value Features for Controllable Adversarial Attacks on LVLMs25 Nov
  2. 2024Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks24 Nov
  3. 2023Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models26 Jul

Other references · 11

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  6. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  7. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  8. 2023Physically Adversarial Infrared Patches with Learnable Shapes and Locationscited by 38
  9. 2022HOTCOLD Block: Fooling Thermal Infrared Detectors with a Novel Wearable Designcited by 27
  10. 2023Adversarial Infrared Curves: An Attack on Infrared Pedestrian Detectors in the Physical Worldcited by 10
  11. 2025IF-Bench: Benchmarking and Enhancing MLLMs for Infrared Images with Generative Visual Promptingcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026InfraQR: Edge-Placed QR-Inspired Structured Patch Attacks on Infrared Vision-Language Models
  2. 2026Thermo-VL: Extending Vision-Language Models to Thermal Infrared Perception
  3. 2025RGB-Th-Bench: A Dense benchmark for Visual-Thermal Understanding of Vision Language Models
  4. 2026ThermEval: A Structured Benchmark for Evaluation of Vision-Language Models on Thermal Imagery
  5. 2026Revealing Physical-World Semantic Vulnerabilities: Universal Adversarial Patch for Infrared Vision-Language Models
  6. 2026InfraPatch: Cross-Task Targeted Grayscale Patch Attacks on Infrared-Adapted Vision-Language Models
  7. 2026T-CLIP: Enabling Thermal Perception for Contrastive Language-Image Pretraining
  8. 2026MonoIR-RS: Infrared Remote Sensing Vision-Language Learning with CLIP and VLM Adaptation