# An Audio-Visual Fusion Emotion Generation Model Based on Neuroanatomical Alignment

arXiv: https://arxiv.org/abs/2503.16454 · HTML: https://mlsanity.com/paper/2503.16454/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 51 references that are in the map.
- First posted on arXiv 21 Feb 2025.

## Builds on (most cited first, 7 of 7)

- [Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory](https://mlsanity.com/paper/1704.01074/) (2017)
- [Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs](https://mlsanity.com/paper/1909.06161/) (2019)
- [A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances](https://mlsanity.com/paper/2203.06935/) (2022)
- [Multi-target DoA Estimation with an Audio-visual Fusion Mechanism](https://mlsanity.com/paper/2105.06107/) (2021)
- [Depression Recognition using Remote Photoplethysmography from Facial Videos](https://mlsanity.com/paper/2206.04399/) (2022)
- [Multimodal Audio-Visual Information Fusion using Canonical-Correlated Graph Neural Network for Energy-Efficient Speech Enhancement](https://mlsanity.com/paper/2202.04528/) (2022)
- [Predicting emotion from music videos: exploring the relative contribution of visual and auditory information to affective responses](https://mlsanity.com/paper/2202.10453/) (2022)

## Abstract (arXiv)

In the field of affective computing, traditional methods for generating emotions predominantly rely on deep learning techniques and large-scale emotion datasets. However, deep learning techniques are often complex and difficult to interpret, and standardizing large-scale emotional datasets are difficult and costly to establish. To tackle these challenges, we introduce a novel framework named Audio-Visual Fusion for Brain-like Emotion Learning(AVF-BEL). In contrast to conventional brain-inspired emotion learning methods, this approach improves the audio-visual emotion fusion and generation model through the integration of modular components, thereby enabling more lightweight and interpretable emotion learning and generation processes. The framework simulates the integration of the visual, auditory, and emotional pathways of the brain, optimizes the fusion of emotional features across visual and auditory modalities, and improves upon the traditional Brain Emotional Learning (BEL) model. The experimental results indicate a significant improvement in the similarity of the audio-visual fusion emotion learning generation model compared to single-modality visual and auditory emotion learning and generation model. Ultimately, this aligns with the fundamental phenomenon of heightened emotion generation facilitated by the integrated impact of visual and auditory stimuli. This contribution not only enhances the interpretability and efficiency of affective intelligence but also provides new insights and pathways for advancing affective computing technology. Our source code can be accessed here: https://github.com/OpenHUTB/emotion}{https://github.com/OpenHUTB/emotion.