mlsanityData to 7 Oct 2026

Symphony of Bias: Exploring Gender Associations with Musical Instruments in Multimodal LLMs

arXiv 2607.26355 ↗First posted 29 Jul 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are increasingly embedded in everyday life and widely used for information seeking, raising concerns about their potential to perpetuate social biases and reinforce stereotypes. In this study, we investigate gender bias in LLMs through the lens of their associations with musical instruments. Building on social-science research on the cultural gender-typing of instruments, we introduce Symphony-Bias, a parallel multimodal dataset spanning text, vision, and audio. We evaluate ten multimodal models with diverse architectures and scales across 22 musical instruments, analyzing how they associate each instrument with three gender categories: male, female, non-binary, across three modalities: text, vision, audio. Our results show that 92% of instrument-level outcomes align with prior social-science findings, with the harp and drums showing particularly consistent gendered associations across all evaluated models and modalities. We further find that alignment with social stereotypes is weakest in audio, stronger in vision, and strongest in text, suggesting that modality-specific representations can differentially amplify gendered associations with musical instruments.footnoteThe Symphony-Bias dataset will be publicly released upon acceptance of the paper.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 58 references that are in the map. First posted on arXiv 29 Jul 2026.

Builds on

18 of 58 references.

References · 18

  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  2. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  3. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  4. 2017Challenges in Data-to-Document Generationcited by 226
  5. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 196
  6. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  7. 2023SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examplescited by 53
  8. 2023Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implicationscited by 29
  9. 2024Uncovering Bias in Large Vision-Language Models at Scale with Counterfactualscited by 27
  10. 2024Beyond Binary Gender Labels: Revealing Gender Biases in LLMs through Gender-Neutral Name Predictionscited by 17
  11. 2025Biased Tales: Cultural and Topic Bias in Generating Children's Storiescited by 15
  12. 2025AesBiasBench: Evaluating Bias and Alignment in Multimodal Language Models for Personalized Image Aesthetic Assessmentcited by 10
6 more
  1. 2025Quantifying Cognitive Bias Induction in LLM-Generated Contentcited by 6
  2. 2025Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translationcited by 5
  3. 2025The in-context inductive biases of vision-language models differ across modalitiescited by 4
  4. 2025Sports and Women's Sports: Gender Bias in Text Generation with Olympic Datacited by 3
  5. 2025Freeze and Reveal: Exposing Modality Bias in Vision-Language Modelscited by 2
  6. 2025PBBQ: A Persian Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Models
  2. 2023Gender bias and stereotypes in Large Language Models
  3. 2025The MUSE Benchmark: Probing Music Perception and Auditory Relational Reasoning in Audio LLMS
  4. 2025More Women, Same Stereotypes: Unpacking the Gender Bias Paradox in Large Language Models
  5. 2024How far can bias go? Tracing bias from pretraining data to alignment
  6. 2024The Role of Large Language Models in Musicology: Are We Ready to Trust the Machines?
  7. 2024White Men Lead, Black Women Help? Benchmarking and Mitigating Language Agency Social Biases in LLMs
  8. 2024Can LLMs "Reason" in Music? An Evaluation of LLMs' Capability of Music Understanding and Generation