# Differentiable Weighted Finite-State Transducers

arXiv: https://arxiv.org/abs/2010.01003 · HTML: https://mlsanity.com/paper/2010.01003/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2020.

## Built on by (most cited first, 22 of 22)

- [Unsupervised Speech Recognition](https://mlsanity.com/paper/2105.11084/) (2021)
- [Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks](https://mlsanity.com/paper/2105.00573/) (2021)
- [The Foundations of Tokenization: Statistical and Computational Concerns](https://mlsanity.com/paper/2407.11606/) (2024)
- [Bayes risk CTC: Controllable CTC alignment in Sequence-to-Sequence tasks](https://mlsanity.com/paper/2210.07499/) (2022)
- [Differentiable Allophone Graphs for Language-Universal Speech Recognition](https://mlsanity.com/paper/2107.11628/) (2021)
- [The History of Speech Recognition to the Year 2030](https://mlsanity.com/paper/2108.00084/) (2021)
- [Star Temporal Classification: Sequence Classification with Partially Labeled Data](https://mlsanity.com/paper/2201.12208/) (2022)
- [Bypass Temporal Classification: Weakly Supervised Automatic Speech Recognition with Imperfect Transcripts](https://mlsanity.com/paper/2306.01031/) (2023)
- [Powerful and Extensible WFST Framework for RNN-Transducer Losses](https://mlsanity.com/paper/2303.10384/) (2023)
- [Alternative Pseudo-Labeling for Semi-Supervised Automatic Speech Recognition](https://mlsanity.com/paper/2308.06547/) (2023)
- [Sequence Transduction with Graph-based Supervision](https://mlsanity.com/paper/2111.01272/) (2021)
- [Integrating Lattice-Free MMI into End-to-End Speech Recognition](https://mlsanity.com/paper/2203.15614/) (2022)
- [Revisiting the Entropy Semiring for Neural Speech Recognition](https://mlsanity.com/paper/2312.10087/) (2023)
- [Extended Graph Temporal Classification for Multi-Speaker End-to-End ASR](https://mlsanity.com/paper/2203.00232/) (2022)
- [Neural Search Space in Gboard Decoder](https://mlsanity.com/paper/2410.15575/) (2024)
- [RNN-Transducer-based Losses for Speech Recognition on Noisy Targets](https://mlsanity.com/paper/2504.06963/) (2025)
- [Higher-order Derivatives of Weighted Finite-state Machines](https://mlsanity.com/paper/2106.00749/) (2021)
- [Parallel Composition of Weighted Finite-State Transducers](https://mlsanity.com/paper/2110.02848/) (2021)
- [LAST: Scalable Lattice-Based Speech Modelling in JAX](https://mlsanity.com/paper/2304.13134/) (2023)
- [Multi-Abstractive Neural Controller: An Efficient Hierarchical Control Architecture for Interactive Driving](https://mlsanity.com/paper/2305.14797/) (2023)
- [Robust Surgical Phase Recognition From Annotation Efficient Supervision](https://mlsanity.com/paper/2406.18481/) (2024)
- [A Training Criterion with Token-Level Tolerance to Transcription Ambiguity for Automatic Speech Recognition](https://mlsanity.com/paper/2609.30160/) (2026)

## Abstract (arXiv)

We introduce a framework for automatic differentiation with weighted finite-state transducers (WFSTs) allowing them to be used dynamically at training time. Through the separation of graphs from operations on graphs, this framework enables the exploration of new structured loss functions which in turn eases the encoding of prior knowledge into learning algorithms. We show how the framework can combine pruning and back-off in transition models with various sequence-level loss functions. We also show how to learn over the latent decomposition of phrases into word pieces. Finally, to demonstrate that WFSTs can be used in the interior of a deep neural network, we propose a convolutional WFST layer which maps lower-level representations to higher-level representations and can be used as a drop-in replacement for a traditional convolution. We validate these algorithms with experiments in handwriting recognition and speech recognition.