# MFF-EINV2: Multi-scale Feature Fusion across Spectral-Spatial-Temporal Domains for Sound Event Localization and Detection

arXiv: https://arxiv.org/abs/2406.08771 · HTML: https://mlsanity.com/paper/2406.08771/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 26 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Builds on (most cited first, 22 of 22)

- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1807.00129/) (2018)
- [FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement](https://mlsanity.com/paper/2010.15508/) (2020)
- [STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events](https://mlsanity.com/paper/2306.09126/) (2023)
- [FullSubNet+: Channel Attention FullSubNet with Complex Spectrograms for Speech Enhancement](https://mlsanity.com/paper/2203.12188/) (2022)
- [ACCDOA: Activity-Coupled Cartesian Direction of Arrival Representation for Sound Event Localization and Detection](https://mlsanity.com/paper/2010.15306/) (2020)
- [An Improved Event-Independent Network for Polyphonic Sound Event Localization and Detection](https://mlsanity.com/paper/2010.13092/) (2020)
- [Multi-ACCDOA: Localizing and Detecting Overlapping Sounds from the Same Class with Auxiliary Duplicating Permutation Invariant Training](https://mlsanity.com/paper/2110.07124/) (2021)
- [Frequency Dynamic Convolution: Frequency-Adaptive Pattern Recognition for Sound Event Detection](https://mlsanity.com/paper/2203.15296/) (2022)
- [Channel-Attention Dense U-Net for Multichannel Speech Enhancement](https://mlsanity.com/paper/2001.11542/) (2020)
- [STARSS22: A dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events](https://mlsanity.com/paper/2206.01948/) (2022)
- [FN-SSL: Full-Band and Narrow-Band Fusion for Sound Source Localization](https://mlsanity.com/paper/2305.19610/) (2023)
- [Event-Independent Network for Polyphonic Sound Event Localization and Detection](https://mlsanity.com/paper/2010.00140/) (2020)
- [McNet: Fuse Multiple Cues for Multichannel Speech Enhancement](https://mlsanity.com/paper/2211.08872/) (2022)
- [CST-former: Transformer with Channel-Spectro-Temporal Attention for Sound Event Localization and Detection](https://mlsanity.com/paper/2312.12821/) (2023)
- [A Track-Wise Ensemble Event Independent Network for Polyphonic Sound Event Localization and Detection](https://mlsanity.com/paper/2203.10228/) (2022)
- [Sound Event Localization and Detection for Real Spatial Sound Scenes: Event-Independent Network and Data Augmentation Chains](https://mlsanity.com/paper/2209.01802/) (2022)
- [Frequency & Channel Attention for Computationally Efficient Sound Event Detection](https://mlsanity.com/paper/2306.11277/) (2023)
- [A Multi-grained based Attention Network for Semi-supervised Sound Event Detection](https://mlsanity.com/paper/2206.10175/) (2022)
- [Divided spectro-temporal attention for sound event localization and detection in real scenes for DCASE2023 challenge](https://mlsanity.com/paper/2306.02591/) (2023)

## Built on by (most cited first, 1 of 1)

- [SelectTSL: Prompt-Guided Selective Target Sound Localization in Complex Scenarios](https://mlsanity.com/paper/2607.02343/) (2026)

## Abstract (arXiv)

Sound Event Localization and Detection (SELD) involves detecting and localizing sound events using multichannel sound recordings. Previously proposed Event-Independent Network V2 (EINV2) has achieved outstanding performance on SELD. However, it still faces challenges in effectively extracting features across spectral, spatial, and temporal domains. This paper proposes a three-stage network structure named Multi-scale Feature Fusion (MFF) module to fully extract multi-scale features across spectral, spatial, and temporal domains. The MFF module utilizes parallel subnetworks architecture to generate multi-scale spectral and spatial features. The TF-Convolution Module is employed to provide multi-scale temporal features. We incorporated MFF into EINV2 and term the proposed method as MFF-EINV2. Experimental results in 2022 and 2023 DCASE challenge task3 datasets show the effectiveness of our MFF-EINV2, which achieves state-of-the-art (SOTA) performance compared to published methods.