# Robust Neural Machine Translation with Joint Textual and Phonetic Embedding

arXiv: https://arxiv.org/abs/1810.06729 · HTML: https://mlsanity.com/paper/1810.06729/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2018.

## Built on by (most cited first, 19 of 19)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections](https://mlsanity.com/paper/2104.04670/) (2021)
- [MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks](https://mlsanity.com/paper/2310.19677/) (2023)
- [Robust Transfer Learning with Pretrained Language Models through Adapters](https://mlsanity.com/paper/2108.02340/) (2021)
- [Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods](https://mlsanity.com/paper/2203.05227/) (2022)
- [Alternative Input Signals Ease Transfer in Multilingual Machine Translation](https://mlsanity.com/paper/2110.07804/) (2021)
- [MergeDistill: Merging Pre-trained Language Models using Distillation](https://mlsanity.com/paper/2106.02834/) (2021)
- [An Approach to Improve Robustness of NLP Systems against ASR Errors](https://mlsanity.com/paper/2103.13610/) (2021)
- [PhonologyBench: Evaluating Phonological Skills of Large Language Models](https://mlsanity.com/paper/2404.02456/) (2024)
- [Addressing the Vulnerability of NMT in Input Perturbations](https://mlsanity.com/paper/2104.09810/) (2021)
- [Pre-training for Spoken Language Understanding with Joint Textual and Phonetic Representation Learning](https://mlsanity.com/paper/2104.10357/) (2021)
- [Improving Multilingual Neural Machine Translation System for Indic Languages](https://mlsanity.com/paper/2209.13279/) (2022)
- [Secoco: Self-Correcting Encoding for Neural Machine Translation](https://mlsanity.com/paper/2108.12137/) (2021)
- [Multilingual Extraction and Categorization of Lexical Collocations with Graph-aware Transformers](https://mlsanity.com/paper/2205.11456/) (2022)
- [Task-guided Disentangled Tuning for Pretrained Language Models](https://mlsanity.com/paper/2203.11431/) (2022)
- [Investigating Lexical Sharing in Multilingual Machine Translation for Indian Languages](https://mlsanity.com/paper/2305.03207/) (2023)
- [Investigating Glyph Phonetic Information for Chinese Spell Checking: What Works and What's Next](https://mlsanity.com/paper/2212.04068/) (2022)
- [Towards Reliable Neural Machine Translation with Consistency-Aware Meta-Learning](https://mlsanity.com/paper/2303.10966/) (2023)
- [An Empirical Study on the Robustness of Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/2405.07673/) (2024)

## Abstract (arXiv)

Neural machine translation (NMT) is notoriously sensitive to noises, but noises are almost inevitable in practice. One special kind of noise is the homophone noise, where words are replaced by other words with similar pronunciations. We propose to improve the robustness of NMT to homophone noises by 1) jointly embedding both textual and phonetic information of source sentences, and 2) augmenting the training dataset with homophone noises. Interestingly, to achieve better translation quality and more robustness, we found that most (though not all) weights should be put on the phonetic rather than textual information. Experiments show that our method not only significantly improves the robustness of NMT to homophone noises, but also surprisingly improves the translation quality on some clean test sets.