# CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models

arXiv: https://arxiv.org/abs/2310.08753 · HTML: https://mlsanity.com/paper/2310.08753/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 47 of its 59 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)

## Builds on (most cited first, 30 of 47)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [VGGSound: A Large-scale Audio-Visual Dataset](https://mlsanity.com/paper/2004.14368/) (2020)
- [When and why vision-language models behave like bags-of-words, and what to do about it?](https://mlsanity.com/paper/2210.01936/) (2022)
- [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://mlsanity.com/paper/2301.12503/) (2023)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality](https://mlsanity.com/paper/2204.03162/) (2022)
- [MusicLM: Generating Music From Text](https://mlsanity.com/paper/2301.11325/) (2023)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [Clotho: An Audio Captioning Dataset](https://mlsanity.com/paper/1910.09387/) (2019)
- [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://mlsanity.com/paper/1611.01368/) (2016)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models](https://mlsanity.com/paper/2301.12661/) (2023)
- [Pengi: An Audio Language Model for Audio Tasks](https://mlsanity.com/paper/2305.11834/) (2023)
- [Colorless green recurrent networks dream hierarchically](https://mlsanity.com/paper/1803.11138/) (2018)
- [Wav2CLIP: Learning Robust Audio Representations From CLIP](https://mlsanity.com/paper/2110.11499/) (2021)
- [CREPE: Can Vision-Language Foundation Models Reason Compositionally?](https://mlsanity.com/paper/2212.07796/) (2022)
- [Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little](https://mlsanity.com/paper/2104.06644/) (2021)
- [HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection](https://mlsanity.com/paper/2202.00874/) (2022)
- [A Systematic Assessment of Syntactic Generalization in Neural Language Models](https://mlsanity.com/paper/2005.03692/) (2020)

## Built on by (most cited first, 30 of 43)

- [MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2410.19168/) (2024)
- [GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities](https://mlsanity.com/paper/2406.11768/) (2024)
- [Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities](https://mlsanity.com/paper/2503.03983/) (2025)
- [MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix](https://mlsanity.com/paper/2505.13032/) (2025)
- [A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Models](https://mlsanity.com/paper/2405.09589/) (2024)
- [OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM](https://mlsanity.com/paper/2510.15870/) (2025)
- [SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based on Speech and Audio Information](https://mlsanity.com/paper/2505.13237/) (2025)
- [Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2505.15957/) (2025)
- [Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoning](https://mlsanity.com/paper/2410.16130/) (2024)
- [Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation](https://mlsanity.com/paper/2410.10676/) (2024)
- [Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music](https://mlsanity.com/paper/2604.10905/) (2026)
- [SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models](https://mlsanity.com/paper/2508.02018/) (2025)
- [ReCLAP: Improving Zero Shot Audio Classification by Describing Sounds](https://mlsanity.com/paper/2409.09213/) (2024)
- [Improving Musical Accompaniment Co-creation via Diffusion Transformers](https://mlsanity.com/paper/2410.23005/) (2024)
- [BLAB: Brutally Long Audio Bench](https://mlsanity.com/paper/2505.03054/) (2025)
- [Benchmarking and Confidence Evaluation of LALMs For Temporal Reasoning](https://mlsanity.com/paper/2505.13115/) (2025)
- [AQAScore: Evaluating Semantic Alignment in Text-to-Audio Generation via Audio Question Answering](https://mlsanity.com/paper/2601.14728/) (2026)
- [Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data](https://mlsanity.com/paper/2410.02056/) (2024)
- [From Alignment to Advancement: Bootstrapping Audio-Language Alignment with Synthetic Data](https://mlsanity.com/paper/2505.20166/) (2025)
- [BRACE: A Benchmark for Robust Audio Caption Quality Evaluation](https://mlsanity.com/paper/2512.10403/) (2025)
- [Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Models](https://mlsanity.com/paper/2409.18680/) (2024)
- [ATRI: Mitigating Multilingual Audio Text Retrieval Inconsistencies by Reducing Data Distribution Errors](https://mlsanity.com/paper/2502.14627/) (2025)
- [Enhancing Temporal Understanding in Audio Question Answering for Large Audio Language Models](https://mlsanity.com/paper/2409.06223/) (2024)
- [CCFQA: A Benchmark for Cross-Lingual and Cross-Modal Speech and Text Factuality Evaluation](https://mlsanity.com/paper/2508.07295/) (2025)
- [Audio Flamingo Sound-CoT Technical Report: Improving Chain-of-Thought Reasoning in Sound Understanding](https://mlsanity.com/paper/2508.11818/) (2025)
- [Do Audio-Language Models Understand Linguistic Variations?](https://mlsanity.com/paper/2410.16505/) (2024)
- [RiTTA: Modeling Event Relations in Text-to-Audio Generation](https://mlsanity.com/paper/2412.15922/) (2024)
- [SupCLAP: Controlling Optimization Trajectory Drift in Audio-Text Contrastive Learning with Support Vector Regularization](https://mlsanity.com/paper/2509.21033/) (2025)
- [TSPE: Task-Specific Prompt Ensemble for Improved Zero-Shot Audio Classification](https://mlsanity.com/paper/2501.00398/) (2024)
- [WoW-Bench: Evaluating Fine-Grained Acoustic Perception in Audio-Language Models via Marine Mammal Vocalizations](https://mlsanity.com/paper/2508.20976/) (2025)

## Abstract (arXiv)

A fundamental characteristic of audio is its compositional nature. Audio-language models (ALMs) trained using a contrastive approach (e.g., CLAP) that learns a shared representation between audio and language modalities have improved performance in many downstream applications, including zero-shot audio classification, audio retrieval, etc. However, the ability of these models to effectively perform compositional reasoning remains largely unexplored and necessitates additional research. In this paper, we propose CompA, a collection of two expert-annotated benchmarks with a majority of real-world audio samples, to evaluate compositional reasoning in ALMs. Our proposed CompA-order evaluates how well an ALM understands the order or occurrence of acoustic events in audio, and CompA-attribute evaluates attribute-binding of acoustic events. An instance from either benchmark consists of two audio-caption pairs, where both audios have the same acoustic events but with different compositions. An ALM is evaluated on how well it matches the right audio to the right caption. Using this benchmark, we first show that current ALMs perform only marginally better than random chance, thereby struggling with compositional reasoning. Next, we propose CompA-CLAP, where we fine-tune CLAP using a novel learning method to improve its compositional reasoning abilities. To train CompA-CLAP, we first propose improvements to contrastive training with composition-aware hard negatives, allowing for more focused training. Next, we propose a novel modular contrastive loss that helps the model learn fine-grained compositional understanding and overcomes the acute scarcity of openly available compositional audios. CompA-CLAP significantly improves over all our baseline models on the CompA benchmark, indicating its superior compositional reasoning capabilities.