# NeMo: a toolkit for building AI applications using Neural Modules

arXiv: https://arxiv.org/abs/1909.09577 · HTML: https://mlsanity.com/paper/1909.09577/

## Key facts

- Cited by 235 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2019.

## Built on by (most cited first, 30 of 235)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://mlsanity.com/paper/2206.04658/) (2022)
- [NVILA: Efficient Frontier Visual Language Models](https://mlsanity.com/paper/2412.04468/) (2024)
- [ChipNeMo: Domain-Adapted LLMs for Chip Design](https://mlsanity.com/paper/2311.00176/) (2023)
- [OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset](https://mlsanity.com/paper/2402.10176/) (2024)
- [Generative Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2203.16502/) (2022)
- [Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels](https://mlsanity.com/paper/2303.14307/) (2023)
- [SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF](https://mlsanity.com/paper/2310.05344/) (2023)
- [StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2212.11851/) (2022)
- [HD-EPIC: A Highly-Detailed Egocentric Video Dataset](https://mlsanity.com/paper/2502.04144/) (2025)
- [Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance](https://mlsanity.com/paper/2111.11755/) (2021)
- [Zipformer: A faster and better encoder for automatic speech recognition](https://mlsanity.com/paper/2310.11230/) (2023)
- [SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognition](https://mlsanity.com/paper/2104.02014/) (2021)
- [SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation](https://mlsanity.com/paper/2310.09424/) (2023)
- [EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2406.06185/) (2024)
- [Efficient Sequence Transduction by Jointly Predicting Tokens and Durations](https://mlsanity.com/paper/2304.06795/) (2023)
- [Open-Source Conversational AI with SpeechBrain 1.0](https://mlsanity.com/paper/2407.00463/) (2024)
- [Privacy-Preserving In-Context Learning for Large Language Models](https://mlsanity.com/paper/2305.01639/) (2023)
- [Upcycling Large Language Models into Mixture of Experts](https://mlsanity.com/paper/2410.07524/) (2024)
- [Flextron: Many-in-One Flexible Large Language Model](https://mlsanity.com/paper/2406.10260/) (2024)
- [DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data](https://mlsanity.com/paper/2409.20007/) (2024)
- [NeMo-Aligner: Scalable Toolkit for Efficient Model Alignment](https://mlsanity.com/paper/2405.01481/) (2024)
- [Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Data](https://mlsanity.com/paper/2205.15370/) (2022)
- [SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks](https://mlsanity.com/paper/2212.10525/) (2022)
- [SALM-Duplex: Efficient and Direct Duplex Modeling for Speech-to-Speech Language Model](https://mlsanity.com/paper/2505.15670/) (2025)
- [DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment](https://mlsanity.com/paper/2406.18871/) (2024)
- [NeMo Inverse Text Normalization: From Development To Production](https://mlsanity.com/paper/2104.05055/) (2021)
- [Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models](https://mlsanity.com/paper/2406.10507/) (2024)
- [Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generation](https://mlsanity.com/paper/2407.09886/) (2024)

## Abstract (arXiv)

NeMo (Neural Modules) is a Python framework-agnostic toolkit for creating AI applications through re-usability, abstraction, and composition. NeMo is built around neural modules, conceptual blocks of neural networks that take typed inputs and produce typed outputs. Such modules typically represent data layers, encoders, decoders, language models, loss functions, or methods of combining activations. NeMo makes it easy to combine and re-use these building blocks while providing a level of semantic correctness checking via its neural type system. The toolkit comes with extendable collections of pre-built modules for automatic speech recognition and natural language processing. Furthermore, NeMo provides built-in support for distributed training and mixed precision on latest NVIDIA GPUs. NeMo is open-source https://github.com/NVIDIA/NeMo