mlsanityData to 7 Oct 2026

Building and curating conversational corpora for diversity-aware language science and technology

arXiv 2203.03399 ↗First posted 7 Mar 2022cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

We present an analysis pipeline and best practice guidelines for building and curating corpora of everyday conversation in diverse languages. Surveying language documentation corpora and other resources that cover 67 languages and varieties from 28 phyla, we describe the compilation and curation process, specify minimal properties of a unified format for interactional data, and develop methods for quality control that take into account turn-taking and timing. Two case studies show the broad utility of conversational data for (i) charting human interactional infrastructure and (ii) tracing challenges and opportunities for current ASR solutions. Linguistically diverse conversational corpora can provide new insights for the language sciences and stronger empirical foundations for language technology.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 13 of its 144 references that are in the map. First posted on arXiv 7 Mar 2022.

Builds on

13 of 144 references.

References · 13

  1. 2018Datasheets for Datasetscited by 1,371
  2. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  3. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  4. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  5. 2020ConvoKit: A Toolkit for the Analysis of Conversationscited by 71
  6. 2021Including Signed Languages in Natural Language Processingcited by 67
  7. 2021Changing the World by Changing the Datacited by 51
  8. 2021Just What do You Think You're Doing, Dave?' A Checklist for Responsible Data Use in NLPcited by 47
  9. 2017Scattertext: a Browser-Based Tool for Visualizing how Corpora Differcited by 22
  10. 2019Disfluencies and Human Speech Transcription Errorscited by 21
  11. 2020Towards decolonising computational sciencescited by 17
  12. 2021A Toolbox for Construction and Analysis of Speech Datasetscited by 5
1 more
  1. 2021What shall we do with an hour of data? Speech recognition for the un- and under-served languages of Common Voicecited by 5

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 1.

2 papers.

  1. 2023The timing bottleneck: Why timing and overlap are mission-critical for conversational user interfaces, speech recognition and dialogue systemscited by 9
  2. 2025Investigating the Representation of Backchannels and Fillers in Fine-tuned Language Modelscited by 1