# Lattice-Based Transformer Encoder for Neural Machine Translation

arXiv: https://arxiv.org/abs/1906.01282 · HTML: https://mlsanity.com/paper/1906.01282/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 5 of 5)

- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Relative Positional Encoding for Transformers with Linear Complexity](https://mlsanity.com/paper/2105.08399/) (2021)
- [Lattention: Lattice-attention in ASR rescoring](https://mlsanity.com/paper/2111.10157/) (2021)
- [DeepLOC: Deep Learning-based Bone Pathology Localization and Classification in Wrist X-ray Images](https://mlsanity.com/paper/2308.12727/) (2023)
- [Confusion2vec 2.0: Enriching Ambiguous Spoken Language Representations with Subwords](https://mlsanity.com/paper/2102.02270/) (2021)

## Abstract (arXiv)

Neural machine translation (NMT) takes deterministic sequences for source representations. However, either word-level or subword-level segmentations have multiple choices to split a source sequence with different word segmentors or different subword vocabulary sizes. We hypothesize that the diversity in segmentations may affect the NMT performance. To integrate different segmentations with the state-of-the-art NMT model, Transformer, we propose lattice-based encoders to explore effective word or subword representation in an automatic way during training. We propose two methods: 1) lattice positional encoding and 2) lattice-aware self-attention. These two methods can be used together and show complementary to each other to further improve translation performance. Experiment results show superiorities of lattice-based encoders in word-level and subword-level representations over conventional Transformer encoder.