mlsanityData to 7 Oct 2026

A Multi-Purpose Audio-Visual Corpus for Multi-Modal Persian Speech Recognition: the Arman-AV Dataset

arXiv 2301.10180 ↗First posted 21 Jan 2023cs.CL cs.SD eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

In recent years, significant progress has been made in automatic lip reading. But these methods require large-scale datasets that do not exist for many low-resource languages. In this paper, we have presented a new multipurpose audio-visual dataset for Persian. This dataset consists of almost 220 hours of videos with 1760 corresponding speakers. In addition to lip reading, the dataset is suitable for automatic speech recognition, audio-visual speech recognition, and speaker recognition. Also, it is the first large-scale lip reading dataset in Persian. A baseline method was provided for each mentioned task. In addition, we have proposed a technique to detect visemes (a visual equivalent of a phoneme) in Persian. The visemes obtained by this method increase the accuracy of the lip reading task by 7% relatively compared to the previously proposed visemes, which can be applied to other languages as well.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 29 of its 54 references that are in the map. First posted on arXiv 21 Jan 2023.

Builds on

29 of 54 references.

References · 29

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  3. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  4. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  5. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
  6. 2018Deep Audio-Visual Speech Recognitioncited by 318
  7. 2018LRS3-TED: a large-scale dataset for visual speech recognitioncited by 301
  8. 2022Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Predictioncited by 277
  9. 2016Lip Reading Sentences in the Wildcited by 226
  10. 2021End-to-end Audio-visual Speech Recognition with Conformerscited by 147
  11. 2017S3FD: Single Shot Scale-invariant Face Detectorcited by 142
  12. 2021Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detectioncited by 110
17 more
  1. 2019ASR is all you need: cross-modal distillation for lip readingcited by 85
  2. 2019AVA-ActiveSpeaker: An Audio-Visual Dataset for Active Speaker Detectioncited by 81
  3. 2018Audio-Visual Speech Recognition With A Hybrid CTC/Attention Architecturecited by 61
  4. 2021TitaNet: Neural Model for speaker representation with 1D Depth-wise separable convolutions and global contextcited by 61
  5. 2021Sub-word Level Lip Reading With Visual Attentioncited by 60
  6. 2018Large-Scale Visual Speech Recognitioncited by 57
  7. 2018LRW-1000: A Naturally-Distributed Large-Scale Benchmark for Lip Reading in the Wildcited by 51
  8. 2019CN-CELEB: a challenging Chinese speaker recognition datasetcited by 47
  9. 2019Hearing Lips: Improving Lip Reading by Distilling Speech Recognizerscited by 39
  10. 2022Distinguishing Homophenes Using Multi-Head Visual-Audio Memory for Lip Readingcited by 39
  11. 2019A Cascade Sequence-to-Sequence Model for Chinese Mandarin Lip Readingcited by 25
  12. 2022Training Strategies for Improved Lip-readingcited by 23
  13. 2021Look Who's Talking: Active Speaker Detection in the Wildcited by 21
  14. 2017Towards Estimating the Upper Bound of Visual-Speech Recognition: The Visual Lip-Reading Feasibility Databasecited by 13
  15. 2022A Multimodal German Dataset for Automatic Lip Reading Systems and Transfer Learningcited by 8
  16. 2021Lip reading using external viseme decodingcited by 5
  17. 2021LRWR: Large-Scale Benchmark for Lip Reading in Russian languagecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 1; 2026: 1.

4 papers.

  1. 2023Leveraging Visemes for Better Visual Speech Representation and Lip Readingcited by 2
  2. 2024AnnoTheia: A Semi-Automatic Annotation Toolkit for Audio-Visual Speech Technologiescited by 1
  3. 2025LRW-Persian: Lip-reading in the Wild Dataset for Persian Languagecited by 0
  4. 2026The Annotation Bottleneck in Persian Text NLP: Persian as an Annotation-Scarce Languagecited by 0