# Variational Connectionist Temporal Classification for Order-Preserving Sequence Modeling

arXiv: https://arxiv.org/abs/2309.11983 · HTML: https://mlsanity.com/paper/2309.11983/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 16 of 16)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces](https://mlsanity.com/paper/1805.07467/) (2018)
- [Variational Attention for Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.08207/) (2017)
- [Deep Recurrent Generative Decoder for Abstractive Text Summarization](https://mlsanity.com/paper/1708.00625/) (2017)
- [Deep Graph Random Process for Relational-Thinking-Based Speech Recognition](https://mlsanity.com/paper/2007.02126/) (2020)
- [Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition](https://mlsanity.com/paper/2005.00572/) (2020)
- [Order-Preserving Abstractive Summarization for Spoken Content Based on Connectionist Temporal Classification](https://mlsanity.com/paper/1709.05475/) (2017)

## Built on by (most cited first, 1 of 1)

- [A Joint Spectro-Temporal Relational Thinking Based Acoustic Modeling Framework](https://mlsanity.com/paper/2409.15357/) (2024)

## Abstract (arXiv)

Connectionist temporal classification (CTC) is commonly adopted for sequence modeling tasks like speech recognition, where it is necessary to preserve order between the input and target sequences. However, CTC is only applied to deterministic sequence models, where the latent space is discontinuous and sparse, which in turn makes them less capable of handling data variability when compared to variational models. In this paper, we integrate CTC with a variational model and derive loss functions that can be used to train more generalizable sequence models that preserve order. Specifically, we derive two versions of the novel variational CTC based on two reasonable assumptions, the first being that the variational latent variables at each time step are conditionally independent; and the second being that these latent variables are Markovian. We show that both loss functions allow direct optimization of the variational lower bound for the model log-likelihood, and present computationally tractable forms for implementing them.