# RASR2: The RWTH ASR Toolkit for Generic Sequence-to-sequence Speech Recognition

arXiv: https://arxiv.org/abs/2305.17782 · HTML: https://mlsanity.com/paper/2305.17782/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 44 references that are in the map.
- First posted on arXiv 28 May 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [End-to-End Attention-based Large Vocabulary Speech Recognition](https://mlsanity.com/paper/1508.04395/) (2015)
- [Hybrid Autoregressive Transducer (hat)](https://mlsanity.com/paper/2003.07705/) (2020)
- [Language Modeling with Deep Transformers](https://mlsanity.com/paper/1905.04226/) (2019)
- [RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognition](https://mlsanity.com/paper/1805.05225/) (2018)
- [Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions](https://mlsanity.com/paper/1904.02619/) (2019)
- [Investigating Methods to Improve Language Model Integration for Attention-based Encoder-Decoder ASR Models](https://mlsanity.com/paper/2104.05544/) (2021)
- [Flashlight: Enabling Innovation in Tools for Machine Learning](https://mlsanity.com/paper/2201.12465/) (2022)
- [Phoneme Based Neural Transducer for Large Vocabulary Speech Recognition](https://mlsanity.com/paper/2010.16368/) (2020)
- [Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Network](https://mlsanity.com/paper/2104.11127/) (2021)
- [Internal Language Model Adaptation with Text-Only Data for End-to-End Speech Recognition](https://mlsanity.com/paper/2110.05354/) (2021)
- [On Language Model Integration for RNN Transducer based Speech Recognition](https://mlsanity.com/paper/2110.06841/) (2021)
- [Length bias in Encoder Decoder Models and a Case for Global Conditioning](https://mlsanity.com/paper/1606.03402/) (2016)
- [Efficient Training of Neural Transducer for Speech Recognition](https://mlsanity.com/paper/2204.10586/) (2022)
- [An investigation of phone-based subword units for end-to-end speech recognition](https://mlsanity.com/paper/2004.04290/) (2020)
- [LSTM Language Models for LVCSR in First-Pass Decoding and Lattice-Rescoring](https://mlsanity.com/paper/1907.01030/) (2019)
- [Acoustic Data-Driven Subword Modeling for End-to-End Speech Recognition](https://mlsanity.com/paper/2104.09106/) (2021)
- [Equivalence of Segmental and Neural Transducer Modeling: A Proof of Concept](https://mlsanity.com/paper/2104.06104/) (2021)
- [Robust Beam Search for Encoder-Decoder Attention Based Speech Recognition without Length Bias](https://mlsanity.com/paper/2005.09265/) (2020)
- [HMM vs. CTC for Automatic Speech Recognition: Comparison Based on Full-Sum Training from Scratch](https://mlsanity.com/paper/2210.09951/) (2022)
- [Full-Sum Decoding for Hybrid HMM based Speech Recognition using LSTM Language Model](https://mlsanity.com/paper/2004.00967/) (2020)

## Built on by (most cited first, 3 of 3)

- [Regularizing Learnable Feature Extraction for Automatic Speech Recognition](https://mlsanity.com/paper/2506.09804/) (2025)
- [On the Relation between Internal Language Model and Sequence Discriminative Training for Neural Transducers](https://mlsanity.com/paper/2309.14130/) (2023)
- [Investigating the Effect of Language Models in Sequence Discriminative Training for Neural Transducers](https://mlsanity.com/paper/2310.07345/) (2023)

## Abstract (arXiv)

Modern public ASR tools usually provide rich support for training various sequence-to-sequence (S2S) models, but rather simple support for decoding open-vocabulary scenarios only. For closed-vocabulary scenarios, public tools supporting lexical-constrained decoding are usually only for classical ASR, or do not support all S2S models. To eliminate this restriction on research possibilities such as modeling unit choice, we present RASR2 in this work, a research-oriented generic S2S decoder implemented in C++. It offers a strong flexibility/compatibility for various S2S models, language models, label units/topologies and neural network architectures. It provides efficient decoding for both open- and closed-vocabulary scenarios based on a generalized search framework with rich support for different search modes and settings. We evaluate RASR2 with a wide range of experiments on both switchboard and Librispeech corpora. Our source code is public online.