# A Systematic Study of Cross-Modal Typographic Attacks on Audio-Visual Reasoning

arXiv: https://arxiv.org/abs/2604.03995 · HTML: https://mlsanity.com/paper/2604.03995/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 5 Apr 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)

## Builds on (most cited first, 17 of 17)

- [PandaGPT: One Model To Instruction-Follow Them All](https://mlsanity.com/paper/2305.16355/) (2023)
- [WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs](https://mlsanity.com/paper/2502.04326/) (2025)
- [Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding](https://mlsanity.com/paper/1808.05665/) (2018)
- [AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models](https://mlsanity.com/paper/2410.18325/) (2024)
- [Voxtral](https://mlsanity.com/paper/2507.13264/) (2025)
- [MLLMs are Deeply Affected by Modality Bias](https://mlsanity.com/paper/2505.18657/) (2025)
- [Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks](https://mlsanity.com/paper/2402.00626/) (2024)
- [Multilingual and Multi-Accent Jailbreaking of Audio LLMs](https://mlsanity.com/paper/2504.01094/) (2025)
- [Safeguarding Vision-Language Models Against Patched Visual Prompt Injectors](https://mlsanity.com/paper/2405.10529/) (2024)
- [Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Models](https://mlsanity.com/paper/2501.13772/) (2025)
- [Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection](https://mlsanity.com/paper/2408.03554/) (2024)
- [Exploring Typographic Visual Prompts Injection Threats in Cross-Modality Generation Models](https://mlsanity.com/paper/2503.11519/) (2025)
- [Dyslexify: A Mechanistic Defense Against Typographic Attacks in CLIP](https://mlsanity.com/paper/2508.20570/) (2025)
- [Now You Hear Me: Audio Narrative Attacks Against Large Audio-Language Models](https://mlsanity.com/paper/2601.23255/) (2026)
- [Some Modalities are More Equal Than Others: Decoding and Architecting Multimodal Integration in MLLMs](https://mlsanity.com/paper/2511.22826/) (2025)
- [Physical Prompt Injection Attacks on Large Vision-Language Models](https://mlsanity.com/paper/2601.17383/) (2026)
- [SLANT: Spurious Logo ANalysis Toolkit](https://mlsanity.com/paper/2406.01449/) (2024)

## Abstract (arXiv)

As audio-visual multi-modal large language models (MLLMs) are increasingly deployed in safety-critical applications, understanding their cross-modal vulnerabilities is essential. To this end, we introduce Multi-Modal Typography, a systematic study examining how perturbations added to speech and on-screen text impact audio-visual reasoning of closed- and open-source models. For instance, on the WorldSense benchmark, the attack success rate (ASR) against Gemini 3.7 Flash increased from 9.53% to 43.41%, highlighting its vulnerability. We analyze the cross-modal effects of unimodal and multimodal attacks and evaluate how specific properties of the attacks impact the overall multi-modal reasoning. By applying fine-tuning as a mitigation, we lower the ASR of Qwen2.5-Omni-7B on WorldSense from 60.6% to 52.0%. Our findings across multi-modal reasoning tasks, content moderation, and offline robot task selection benchmarks establish multi-modal typography as a critical and underexplored attack strategy in multi-modal reasoning. Code and data will be made publicly available.