mlsanityData to 14 Oct 2026

AIBA: Attention-based Instrument Band Alignment for Text-to-Audio Diffusion

arXiv 2509.20891 ↗First posted 25 Sep 2025cs.SDcited by 0

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

We present AIBA (Attention-In-Band Alignment), a lightweight, training-free pipeline to quantify where text-to-audio diffusion models attend on the time-frequency (T-F) plane. AIBA (i) hooks cross-attention at inference to record attention probabilities without modifying weights; (ii) projects them to fixed-size mel grids that are directly comparable to audio energy; and (iii) scores agreement with instrument-band ground truth via interpretable metrics (T-F IoU/AP, frequency-profile correlation, and a pointing game). On Slakh2100 with an AudioLDM2 backbone, AIBA reveals consistent instrument-dependent trends (e.g., bass favoring low bands) and achieves high precision with moderate recall.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 20 of its 23 references that are in the map. First posted on arXiv 25 Sep 2025.

Builds on

20 of 23 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining10 Aug
  2. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models30 Jan
  3. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Models29 Jan

Other references · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2022Classifier-Free Diffusion Guidancecited by 5,847
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  8. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  9. 2019Are Sixteen Heads Really Better than One?cited by 816
  10. 2020Quantifying Attention Flow in Transformerscited by 775
  11. 2019Attention is not not Explanationcited by 636
  12. 2023Simple and Controllable Music Generationcited by 488
5 more
  1. 2022AudioGen: Textually Guided Audio Generationcited by 295
  2. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 236
  3. 2021Efficient Training of Audio Transformers with Patchoutcited by 226
  4. 2019Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantitycited by 86
  5. 2021Towards Interpretable and Transferable Speech Emotion Recognition: Latent Representation Based Analysis of Features, Methods and Corporacited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TADA! Tuning Audio Diffusion Models through Activation Steering
  2. 2026SoundWeaver: Semantic Warm-Starting for Text-to-Audio Diffusion Serving
  3. 2026Improving Text-to-Music Generation with Human Preference Rewards
  4. 2026Inside the Latent Flow: Causal Deciphering of Attention Dynamics in Audio Separation Foundation Models
  5. 2024Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation
  6. 2025Melodia: Training-Free Music Editing Guided by Attention Probing in Diffusion Models
  7. 2025SAMUeL: Efficient Vocal-Conditioned Music Generation via Soft Alignment Attention and Latent Diffusion
  8. 2024Improving Musical Accompaniment Co-creation via Diffusion Transformers