# Symphony of Bias: Exploring Gender Associations with Musical Instruments in Multimodal LLMs

arXiv: https://arxiv.org/abs/2607.26355 · HTML: https://mlsanity.com/paper/2607.26355/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 58 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Builds on (most cited first, 18 of 18)

- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Challenges in Data-to-Document Generation](https://mlsanity.com/paper/1707.08052/) (2017)
- ["Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters](https://mlsanity.com/paper/2310.09219/) (2023)
- [Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies](https://mlsanity.com/paper/2108.12084/) (2021)
- [SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples](https://mlsanity.com/paper/2312.00825/) (2023)
- [Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implications](https://mlsanity.com/paper/2307.09162/) (2023)
- [Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals](https://mlsanity.com/paper/2405.20152/) (2024)
- [Beyond Binary Gender Labels: Revealing Gender Biases in LLMs through Gender-Neutral Name Predictions](https://mlsanity.com/paper/2407.05271/) (2024)
- [Biased Tales: Cultural and Topic Bias in Generating Children's Stories](https://mlsanity.com/paper/2509.07908/) (2025)
- [AesBiasBench: Evaluating Bias and Alignment in Multimodal Language Models for Personalized Image Aesthetic Assessment](https://mlsanity.com/paper/2509.11620/) (2025)
- [Quantifying Cognitive Bias Induction in LLM-Generated Content](https://mlsanity.com/paper/2507.03194/) (2025)
- [The in-context inductive biases of vision-language models differ across modalities](https://mlsanity.com/paper/2502.01530/) (2025)
- [Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation](https://mlsanity.com/paper/2511.21517/) (2025)
- [Sports and Women's Sports: Gender Bias in Text Generation with Olympic Data](https://mlsanity.com/paper/2502.04218/) (2025)
- [Freeze and Reveal: Exposing Modality Bias in Vision-Language Models](https://mlsanity.com/paper/2508.07432/) (2025)
- [PBBQ: A Persian Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models](https://mlsanity.com/paper/2510.19616/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are increasingly embedded in everyday life and widely used for information seeking, raising concerns about their potential to perpetuate social biases and reinforce stereotypes. In this study, we investigate gender bias in LLMs through the lens of their associations with musical instruments. Building on social-science research on the cultural gender-typing of instruments, we introduce Symphony-Bias, a parallel multimodal dataset spanning text, vision, and audio. We evaluate ten multimodal models with diverse architectures and scales across 22 musical instruments, analyzing how they associate each instrument with three gender categories: male, female, non-binary, across three modalities: text, vision, audio. Our results show that 92% of instrument-level outcomes align with prior social-science findings, with the harp and drums showing particularly consistent gendered associations across all evaluated models and modalities. We further find that alignment with social stereotypes is weakest in audio, stronger in vision, and strongest in text, suggesting that modality-specific representations can differentially amplify gendered associations with musical instruments.footnoteThe Symphony-Bias dataset will be publicly released upon acceptance of the paper.