# The Attention Triangle in Audio-Video Models

arXiv: https://arxiv.org/abs/2609.03586 · HTML: https://mlsanity.com/paper/2609.03586/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 3 Sep 2026.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [LTX-Video: Realtime Video Latent Diffusion](https://mlsanity.com/paper/2501.00103/) (2024)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [Training-Free Layout Control with Cross-Attention Guidance](https://mlsanity.com/paper/2304.03373/) (2023)
- [LTX-2: Efficient Joint Audio-Visual Foundation Model](https://mlsanity.com/paper/2601.03233/) (2026)
- [Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment](https://mlsanity.com/paper/2306.08877/) (2023)
- [JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronization](https://mlsanity.com/paper/2503.23377/) (2025)
- [Divide & Bind Your Attention for Improved Generative Semantic Nursing](https://mlsanity.com/paper/2307.10864/) (2023)
- [AV-Link: Temporally-Aligned Diffusion Features for Cross-Modal Audio-Video Generation](https://mlsanity.com/paper/2412.15191/) (2024)
- [FreeFlux: Understanding and Exploiting Layer-Specific Roles in RoPE-Based MMDiT for Versatile Image Editing](https://mlsanity.com/paper/2503.16153/) (2025)
- [MM-LDM: Multi-Modal Latent Diffusion Model for Sounding Video Generation](https://mlsanity.com/paper/2410.01594/) (2024)
- [Attention (as Discrete-Time Markov) Chains](https://mlsanity.com/paper/2507.17657/) (2025)
- [SAVGBench: Benchmarking Spatially Aligned Audio-Video Generation](https://mlsanity.com/paper/2412.13462/) (2024)
- [Be Decisive: Noise-Induced Layouts for Multi-Subject Generation](https://mlsanity.com/paper/2505.21488/) (2025)
- [Image Generation from Contextually-Contradictory Prompts](https://mlsanity.com/paper/2506.01929/) (2025)
- [Do Joint Audio-Video Generation Models Understand Physics?](https://mlsanity.com/paper/2605.07061/) (2026)
- [Addressing Text Embedding Leakage in Diffusion-based Image Editing](https://mlsanity.com/paper/2412.04715/) (2024)
- [VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation](https://mlsanity.com/paper/2608.18607/) (2026)
- [Improving Joint Audio-Video Generation with Cross-Modal Context Learning](https://mlsanity.com/paper/2603.18600/) (2026)
- [MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation](https://mlsanity.com/paper/2604.19679/) (2026)
- [Efficient Audio-Visual Generation via Synchrony-Aware Cross-Modal Sparse Attention](https://mlsanity.com/paper/2608.15522/) (2026)

## Built on by (most cited first, 1 of 1)

- [Soundwich: Video Generation with Layered and Controllable Audio](https://mlsanity.com/paper/2610.00691/) (2026)

## Abstract (arXiv)

Audio-video diffusion models rely on cross-modal attention to coordinate text, sound, and visual content, yet this same mechanism can introduce subtle and systematic semantic leakage. We study these models by probing and analyzing the ``attention triangle,'' comprising the three cross-attention edges connecting the text, audio, and video streams, and examine how semantic information is routed across modalities during generation. Our analysis reveals that routing along the audio-video edge is bidirectional: audio can influence video generation, while video can influence audio generation. This edge is shaped by biases encoded in the model's parameters and emerges as a major contributor to leakage: when prompts are in tension with learned priors, cross-modal interactions may override the intended conditioning and reroute semantics toward visually canonical but incorrect outcomes. These effects suggest that semantic artifacts arise not merely from attention spreading beyond its intended target, but from structured, bias-driven interactions along specific pathways. Building on this perspective, we extract attention-derived signals that expose how semantics are distributed and grounded across modalities, and use them as a diagnostic tool to both analyze and deliberately incur leakage under controlled conditions. This enables us to probe the internal dynamics of cross-modal routing and isolate the role of individual interactions. We further leverage these signals to guide inference-time interventions that encourage more consistent cross-modal alignment. Extensive experiments support our analysis and demonstrate improved semantic grounding while preserving generation quality.