mlsanityData to 14 Oct 2026

Lhotse: a speech data representation library for the modern deep learning ecosystem

arXiv 2110.12561 ↗First posted 25 Oct 2021cs.SD eess.AScited by 33

Not on the map. Why?

Abstract (arXiv)

Speech data is notoriously difficult to work with due to a variety of codecs, lengths of recordings, and meta-data formats. We present Lhotse, a speech data representation library that draws upon lessons learned from Kaldi speech recognition toolkit and brings its concepts into the modern deep learning ecosystem. Lhotse provides a common JSON description format with corresponding Python classes and data preparation recipes for over 30 popular speech corpora. Various datasets can be easily combined together and re-purposed for different tasks. The library handles multi-channel recordings, long recordings, local and cloud storage, lazy and on-the-fly operations amongst other features. We introduce Cut and CutSet concepts, which simplify common data wrangling tasks for audio and help incorporate acoustic context of speech utterances. Finally, we show how Lhotse leverages PyTorch data API abstractions and adopts them to handle speech data for deep learning.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 Oct 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 8; 2025: 10; 2026: 8.

33 papers.

Sort:

By area:

  1. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 80
  2. 2025Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and ASTcited by 60
  3. 2025Granary: Speech Recognition and Translation Dataset in 25 European Languagescited by 22
  4. 2024CR-CTC: Consistency regularization on CTC for improved speech recognitioncited by 17
  5. 2022Fast and parallel decoding for transducercited by 12
  6. 2024The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resourcescited by 11
  7. 2023Updated Corpora and Benchmarks for Long-Form Speech Recognitioncited by 10
  8. 2023Speech collage: code-switched audio generation by collaging monolingual corporacited by 10
  9. 2023Blank-regularized CTC for Frame Skipping in Neural Transducercited by 8
  10. 2025Training and Inference Efficiency of Encoder-Decoder Speech Modelscited by 8
  11. 2022Delay-penalized transducer for low-latency streaming ASRcited by 7
  12. 2024VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuningcited by 6
21 more
  1. 2025Word Level Timestamp Generation for Automatic Speech Recognition and Translationcited by 6
  2. 2026TagSpeech: End-to-End Multi-Speaker ASR and Diarization with Fine-Grained Temporal Groundingcited by 5
  3. 2024EMMeTT: Efficient Multimodal Machine Translation Trainingcited by 4
  4. 2025Revisiting Audio-language Pretraining for Learning General-purpose Audio Representationcited by 4
  5. 2025Efficient Data Selection for Domain Adaptation of ASR Using Pseudo-Labels and Multi-Stage Filteringcited by 3
  6. 2026VoiceChat-TTS: A Low-Latency Continuous Speech Synthesis Model for Interactive Agentscited by 3
  7. 2025Better Semi-supervised Learning for Multi-domain ASR Through Incremental Retraining and Data Filteringcited by 2
  8. 2025Adapting Diarization-Conditioned Whisper for End-to-End Multi-Talker Speech Recognitioncited by 2
  9. 2025CLiFT-ASR: A Cross-Lingual Fine-Tuning Framework for Low-Resource Taiwanese Hokkien Speech Recognitioncited by 2
  10. 2024Fast Streaming Transducer ASR Prototyping via Knowledge Distillation with Whispercited by 1
  11. 2025HENT-SRT: Hierarchical Efficient Neural Transducer with Self-Distillation for Joint Speech Recognition and Translationcited by 1
  12. 2026Towards Comprehensive Semantic Speech Embeddings for Chinese Dialectscited by 1
  13. 2026Unlocking Strong Supervision: A Data-Centric Study of General-Purpose Audio Pre-Training Methodscited by 1
  14. 2026A Native-Reference Coordinate Geometry for L2 Pronunciation Deviation Using Self-Supervised Speech Modelscited by 1
  15. 2023MooseNet: A Trainable Metric for Synthesized Speech with a PLDA Modulecited by 0
  16. 2024Pitch-Aware RNN-T for Mandarin Chinese Mispronunciation Detection and Diagnosiscited by 0
  17. 2024Less is More: Accurate Speech Recognition & Translation without Web-Scale Datacited by 0
  18. 2024ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integrationcited by 0
  19. 2026GetBatch: Distributed Multi-Object Retrieval for ML Data Loadingcited by 0
  20. 2026A Native-Reference Phone-Class Geometry for Second-Language Pronunciation Analysiscited by 0
  21. 2026All In Good Time: Causality-Aware Framework for LLM-Based Simultaneous Speech-to-Speech Translationcited by 0