# A Multi-Purpose Audio-Visual Corpus for Multi-Modal Persian Speech Recognition: the Arman-AV Dataset

arXiv: https://arxiv.org/abs/2301.10180 · HTML: https://mlsanity.com/paper/2301.10180/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 21 Jan 2023.

## Builds on (most cited first, 29 of 29)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations](https://mlsanity.com/paper/1910.05453/) (2019)
- [Deep Audio-Visual Speech Recognition](https://mlsanity.com/paper/1809.02108/) (2018)
- [LRS3-TED: a large-scale dataset for visual speech recognition](https://mlsanity.com/paper/1809.00496/) (2018)
- [Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction](https://mlsanity.com/paper/2201.02184/) (2022)
- [Lip Reading Sentences in the Wild](https://mlsanity.com/paper/1611.05358/) (2016)
- [End-to-end Audio-visual Speech Recognition with Conformers](https://mlsanity.com/paper/2102.06657/) (2021)
- [S3FD: Single Shot Scale-invariant Face Detector](https://mlsanity.com/paper/1708.05237/) (2017)
- [Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection](https://mlsanity.com/paper/2107.06592/) (2021)
- [ASR is all you need: cross-modal distillation for lip reading](https://mlsanity.com/paper/1911.12747/) (2019)
- [AVA-ActiveSpeaker: An Audio-Visual Dataset for Active Speaker Detection](https://mlsanity.com/paper/1901.01342/) (2019)
- [Audio-Visual Speech Recognition With A Hybrid CTC/Attention Architecture](https://mlsanity.com/paper/1810.00108/) (2018)
- [Sub-word Level Lip Reading With Visual Attention](https://mlsanity.com/paper/2110.07603/) (2021)
- [TitaNet: Neural Model for speaker representation with 1D Depth-wise separable convolutions and global context](https://mlsanity.com/paper/2110.04410/) (2021)
- [Large-Scale Visual Speech Recognition](https://mlsanity.com/paper/1807.05162/) (2018)
- [LRW-1000: A Naturally-Distributed Large-Scale Benchmark for Lip Reading in the Wild](https://mlsanity.com/paper/1810.06990/) (2018)
- [CN-CELEB: a challenging Chinese speaker recognition dataset](https://mlsanity.com/paper/1911.01799/) (2019)
- [Hearing Lips: Improving Lip Reading by Distilling Speech Recognizers](https://mlsanity.com/paper/1911.11502/) (2019)
- [Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip Reading](https://mlsanity.com/paper/2204.01725/) (2022)
- [A Cascade Sequence-to-Sequence Model for Chinese Mandarin Lip Reading](https://mlsanity.com/paper/1908.04917/) (2019)
- [Training Strategies for Improved Lip-reading](https://mlsanity.com/paper/2209.01383/) (2022)
- [Look Who's Talking: Active Speaker Detection in the Wild](https://mlsanity.com/paper/2108.07640/) (2021)
- [Towards Estimating the Upper Bound of Visual-Speech Recognition: The Visual Lip-Reading Feasibility Database](https://mlsanity.com/paper/1704.08028/) (2017)
- [A Multimodal German Dataset for Automatic Lip Reading Systems and Transfer Learning](https://mlsanity.com/paper/2202.13403/) (2022)
- [Lip reading using external viseme decoding](https://mlsanity.com/paper/2104.04784/) (2021)
- [LRWR: Large-Scale Benchmark for Lip Reading in Russian language](https://mlsanity.com/paper/2109.06692/) (2021)

## Built on by (most cited first, 4 of 4)

- [Leveraging Visemes for Better Visual Speech Representation and Lip Reading](https://mlsanity.com/paper/2307.10157/) (2023)
- [AnnoTheia: A Semi-Automatic Annotation Toolkit for Audio-Visual Speech Technologies](https://mlsanity.com/paper/2402.13152/) (2024)
- [LRW-Persian: Lip-reading in the Wild Dataset for Persian Language](https://mlsanity.com/paper/2510.22716/) (2025)
- [The Annotation Bottleneck in Persian Text NLP: Persian as an Annotation-Scarce Language](https://mlsanity.com/paper/2608.24698/) (2026)

## Abstract (arXiv)

In recent years, significant progress has been made in automatic lip reading. But these methods require large-scale datasets that do not exist for many low-resource languages. In this paper, we have presented a new multipurpose audio-visual dataset for Persian. This dataset consists of almost 220 hours of videos with 1760 corresponding speakers. In addition to lip reading, the dataset is suitable for automatic speech recognition, audio-visual speech recognition, and speaker recognition. Also, it is the first large-scale lip reading dataset in Persian. A baseline method was provided for each mentioned task. In addition, we have proposed a technique to detect visemes (a visual equivalent of a phoneme) in Persian. The visemes obtained by this method increase the accuracy of the lip reading task by 7% relatively compared to the previously proposed visemes, which can be applied to other languages as well.