# Neural Lattice-to-Sequence Models for Uncertain Inputs

arXiv: https://arxiv.org/abs/1704.00559 · HTML: https://mlsanity.com/paper/1704.00559/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2017.

## Built on by (most cited first, 20 of 20)

- [STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation](https://mlsanity.com/paper/2203.10426/) (2022)
- [Learning Shared Semantic Space for Speech-to-Text Translation](https://mlsanity.com/paper/2105.03095/) (2021)
- [Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translation](https://mlsanity.com/paper/2102.05766/) (2021)
- [RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformer](https://mlsanity.com/paper/2106.04833/) (2021)
- [An evaluation of word-level confidence estimation for end-to-end automatic speech recognition](https://mlsanity.com/paper/2101.05525/) (2021)
- [Regularizing End-to-End Speech Translation with Triangular Decomposition Agreement](https://mlsanity.com/paper/2112.10991/) (2021)
- [MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition](https://mlsanity.com/paper/2303.05309/) (2023)
- [Non-Parametric Domain Adaptation for End-to-End Speech Translation](https://mlsanity.com/paper/2205.11211/) (2022)
- [LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Models](https://mlsanity.com/paper/2407.15415/) (2024)
- [R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents](https://mlsanity.com/paper/2303.00732/) (2023)
- [Attention-based Multi-hypothesis Fusion for Speech Summarization](https://mlsanity.com/paper/2111.08201/) (2021)
- [Lattention: Lattice-attention in ASR rescoring](https://mlsanity.com/paper/2111.10157/) (2021)
- [An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/2308.14482/) (2023)
- [Does Joint Training Really Help Cascaded Speech Translation?](https://mlsanity.com/paper/2210.13700/) (2022)
- [Improving speech translation by fusing speech and text](https://mlsanity.com/paper/2305.14042/) (2023)
- [Representation Purification for End-to-End Speech Translation](https://mlsanity.com/paper/2412.04266/) (2024)
- [Learning Domain Specific Language Models for Automatic Speech Recognition through Machine Translation](https://mlsanity.com/paper/2110.10261/) (2021)
- [StressTransfer: Stress-Aware Speech-to-Speech Translation with Emphasis Preservation](https://mlsanity.com/paper/2510.13194/) (2025)
- [Hystoc: Obtaining word confidences for fusion of end-to-end ASR systems](https://mlsanity.com/paper/2305.12579/) (2023)
- [Structure-Aware Path Inference for Neural Finite State Transducers](https://mlsanity.com/paper/2312.13614/) (2023)

## Abstract (arXiv)

The input to a neural sequence-to-sequence model is often determined by an up-stream system, e.g. a word segmenter, part of speech tagger, or speech recognizer. These up-stream models are potentially error-prone. Representing inputs through word lattices allows making this uncertainty explicit by capturing alternative sequences and their posterior probabilities in a compact form. In this work, we extend the TreeLSTM (Tai et al., 2015) into a LatticeLSTM that is able to consume word lattices, and can be used as encoder in an attentional encoder-decoder model. We integrate lattice posterior scores into this architecture by extending the TreeLSTM's child-sum and forget gates and introducing a bias term into the attention mechanism. We experiment with speech translation lattices and report consistent improvements over baselines that translate either the 1-best hypothesis or the lattice without posterior scores.