# AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenario

arXiv: https://arxiv.org/abs/2104.03603 · HTML: https://mlsanity.com/paper/2104.03603/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2021.

## Built on by (most cited first, 30 of 60)

- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [ADD 2022: the First Audio Deep Synthesis Detection Challenge](https://mlsanity.com/paper/2202.08433/) (2022)
- [Ming-Omni: A Unified Multimodal Model for Perception and Generation](https://mlsanity.com/paper/2506.09344/) (2025)
- [InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions](https://mlsanity.com/paper/2412.09596/) (2024)
- [Open Source MagicData-RAMC: A Rich Annotated Mandarin Conversational(RAMC) Speech Dataset](https://mlsanity.com/paper/2203.16844/) (2022)
- [VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Model](https://mlsanity.com/paper/2505.03739/) (2025)
- [Ming-UniAudio: Speech LLM for Joint Understanding, Generation and Editing with Unified Representation](https://mlsanity.com/paper/2511.05516/) (2025)
- [SpeakerLM: End-to-End Versatile Speaker Diarization and Recognition with Multimodal Large Language Models](https://mlsanity.com/paper/2508.06372/) (2025)
- [VIBEVOICE-ASR Technical Report](https://mlsanity.com/paper/2601.18184/) (2026)
- [3D-Speaker: A Large-Scale Multi-Device, Multi-Distance, and Multi-Dialect Corpus for Speech Representation Disentanglement](https://mlsanity.com/paper/2306.15354/) (2023)
- [Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems](https://mlsanity.com/paper/2409.06656/) (2024)
- [StoryTeller: Improving Long Video Description through Global Audio-Visual Character Identification](https://mlsanity.com/paper/2411.07076/) (2024)
- [ContextASR-Bench: A Massive Contextual Speech Recognition Benchmark](https://mlsanity.com/paper/2507.05727/) (2025)
- [Unveiling the Potential of LLM-Based ASR on Chinese Open-Source Datasets](https://mlsanity.com/paper/2405.02132/) (2024)
- [OpenOmni: Advancing Open-Source Omnimodal Large Language Models with Progressive Multimodal Alignment and Real-Time Self-Aware Emotional Speech Synthesis](https://mlsanity.com/paper/2501.04561/) (2025)
- [Efficient and Robust Speaker Diarization via Structured Pruning of Self-Supervised Models](https://mlsanity.com/paper/2506.18623/) (2025)
- [A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings](https://mlsanity.com/paper/2203.16834/) (2022)
- [Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization](https://mlsanity.com/paper/2305.12927/) (2023)
- [BA-SOT: Boundary-Aware Serialized Output Training for Multi-Talker ASR](https://mlsanity.com/paper/2305.13716/) (2023)
- [VoiceWukong: Benchmarking Deepfake Voice Detection](https://mlsanity.com/paper/2409.06348/) (2024)
- [Leveraging Real Conversational Data for Multi-Channel Continuous Speech Separation](https://mlsanity.com/paper/2204.03232/) (2022)
- [Mamba-based Segmentation Model for Speaker Diarization](https://mlsanity.com/paper/2410.06459/) (2024)
- [The Multimodal Information Based Speech Processing (MISP) 2025 Challenge: Audio-Visual Diarization and Recognition](https://mlsanity.com/paper/2505.13971/) (2025)
- [MOSS Transcribe Diarize Technical Report](https://mlsanity.com/paper/2601.01554/) (2026)
- [The RoyalFlush System of Speech Recognition for M2MeT Challenge](https://mlsanity.com/paper/2202.01614/) (2022)
- [The Volcspeech system for the ICASSP 2022 multi-channel multi-party meeting transcription challenge](https://mlsanity.com/paper/2202.04261/) (2022)
- [Royalflush Speaker Diarization System for ICASSP 2022 Multi-channel Multi-party Meeting Transcription Challenge](https://mlsanity.com/paper/2202.04814/) (2022)
- [The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines](https://mlsanity.com/paper/2208.08042/) (2022)
- [Improving Speaker Diarization using Semantic Information: Joint Pairwise Constraints Propagation](https://mlsanity.com/paper/2309.10456/) (2023)
- [SDBench: A Comprehensive Benchmark Suite for Speaker Diarization](https://mlsanity.com/paper/2507.16136/) (2025)

## Abstract (arXiv)

In this paper, we present AISHELL-4, a sizable real-recorded Mandarin speech dataset collected by 8-channel circular microphone array for speech processing in conference scenario. The dataset consists of 211 recorded meeting sessions, each containing 4 to 8 speakers, with a total length of 120 hours. This dataset aims to bridge the advanced research on multi-speaker processing and the practical application scenario in three aspects. With real recorded meetings, AISHELL-4 provides realistic acoustics and rich natural speech characteristics in conversation such as short pause, speech overlap, quick speaker turn, noise, etc. Meanwhile, accurate transcription and speaker voice activity are provided for each meeting in AISHELL-4. This allows the researchers to explore different aspects in meeting processing, ranging from individual tasks such as speech front-end processing, speech recognition and speaker diarization, to multi-modality modeling and joint optimization of relevant tasks. Given most open source dataset for multi-speaker tasks are in English, AISHELL-4 is the only Mandarin dataset for conversation speech, providing additional value for data diversity in speech community. We also release a PyTorch-based training and evaluation framework as baseline system to promote reproducible research in this field.