# MMCosine: Multi-Modal Cosine Loss Towards Balanced Audio-Visual Fine-Grained Learning

arXiv: https://arxiv.org/abs/2303.05338 · HTML: https://mlsanity.com/paper/2303.05338/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2023.

## Built on by (most cited first, 28 of 28)

- [MMPareto: Boosting Multimodal Learning with Innocent Unimodal Assistance](https://mlsanity.com/paper/2405.17730/) (2024)
- [Multimodal Fusion on Low-quality Data: A Comprehensive Survey](https://mlsanity.com/paper/2404.18947/) (2024)
- [Enhancing multimodal cooperation via sample-level modality valuation](https://mlsanity.com/paper/2309.06255/) (2023)
- [Improving Multimodal Learning with Multi-Loss Gradient Modulation](https://mlsanity.com/paper/2405.07930/) (2024)
- [Improving Multimodal Learning via Imbalanced Learning](https://mlsanity.com/paper/2507.10203/) (2025)
- [Quantifying and Enhancing Multi-modal Robustness with Modality Preference](https://mlsanity.com/paper/2402.06244/) (2024)
- [Boosting Multimodal Learning via Disentangled Gradient Learning](https://mlsanity.com/paper/2507.10213/) (2025)
- [Balancing Multimodal Training Through Game-Theoretic Regularization](https://mlsanity.com/paper/2411.07335/) (2024)
- [BalanceBenchmark: A Survey for Multimodal Imbalance Learning](https://mlsanity.com/paper/2502.10816/) (2025)
- [Improving Multimodal Learning Balance and Sufficiency through Data Remixing](https://mlsanity.com/paper/2506.11550/) (2025)
- [Unbiased Dynamic Multimodal Fusion](https://mlsanity.com/paper/2603.19681/) (2026)
- [Path-GPTOmic: A Balanced Multi-modal Learning Framework for Survival Outcome Prediction](https://mlsanity.com/paper/2403.11375/) (2024)
- [Robust Cross-Modal Knowledge Distillation for Unconstrained Videos](https://mlsanity.com/paper/2304.07775/) (2023)
- [eMotions: A Large-Scale Dataset and Audio-Visual Fusion Network for Emotion Analysis in Short-form Videos](https://mlsanity.com/paper/2508.06902/) (2025)
- [Multimodal Classification via Total Correlation Maximization](https://mlsanity.com/paper/2602.13015/) (2026)
- [Shaping Initial State Prevents Modality Competition in Multi-modal Fusion: A Two-stage Scheduling Framework via Fast Partial Information Decomposition](https://mlsanity.com/paper/2509.20840/) (2025)
- [Revisit Modality Imbalance at the Decision Layer](https://mlsanity.com/paper/2510.14411/) (2025)
- [Modality Dominance-Aware Optimization for Embodied RGB-Infrared Perception](https://mlsanity.com/paper/2601.00598/) (2026)
- [BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Rates](https://mlsanity.com/paper/2603.19718/) (2026)
- [PDMP: Rethinking Balanced Multimodal Learning via Performance-Dominant Modality Prioritization](https://mlsanity.com/paper/2604.05773/) (2026)
- [More than Vanilla Fusion: a Simple, Decoupling-free, Attention Module for Multimodal Fusion Based on Signal Theory](https://mlsanity.com/paper/2312.07212/) (2023)
- [What Happens Without Background? Constructing Foreground-Only Data for Fine-Grained Tasks](https://mlsanity.com/paper/2408.01998/) (2024)
- [Mitigating Sample-Level Imbalance via Probabilistic Separation for Adaptive Multimodal Fusion](https://mlsanity.com/paper/2510.21797/) (2025)
- [Multi-Domain EEG Representation Learning with Orthogonal Mapping and Attention-based Fusion for Cognitive Load Classification](https://mlsanity.com/paper/2511.12394/) (2025)
- [Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognition](https://mlsanity.com/paper/2605.21565/) (2026)
- [CAT-GS: Balanced Multimodal Learning via Calibrated Gating and Fusion Surgery](https://mlsanity.com/paper/2608.24947/) (2026)
- [The Illusion of Balanced Multimodal Sentiment Analysis: Beyond the Limits of Optimization-Based Methods](https://mlsanity.com/paper/2609.11247/) (2026)
- [GeoBalance: Geometry-Aware Monitoring and Reconstruction with Asymmetric Optimization for Balanced Multimodal Learning](https://mlsanity.com/paper/2609.23533/) (2026)

## Abstract (arXiv)

Audio-visual learning helps to comprehensively understand the world by fusing practical information from multiple modalities. However, recent studies show that the imbalanced optimization of uni-modal encoders in a joint-learning model is a bottleneck to enhancing the model's performance. We further find that the up-to-date imbalance-mitigating methods fail on some audio-visual fine-grained tasks, which have a higher demand for distinguishable feature distribution. Fueled by the success of cosine loss that builds hyperspherical feature spaces and achieves lower intra-class angular variability, this paper proposes Multi-Modal Cosine loss, MMCosine. It performs a modality-wise L2 normalization to features and weights towards balanced and better multi-modal fine-grained learning. We demonstrate that our method can alleviate the imbalanced optimization from the perspective of weight norm and fully exploit the discriminability of the cosine metric. Extensive experiments prove the effectiveness of our method and the versatility with advanced multi-modal fusion strategies and up-to-date imbalance-mitigating methods.