# Variational Recurrent Neural Machine Translation

arXiv: https://arxiv.org/abs/1801.05119 · HTML: https://mlsanity.com/paper/1801.05119/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2018.

## Built on by (most cited first, 23 of 23)

- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [Learning Language Specific Sub-network for Multilingual Machine Translation](https://mlsanity.com/paper/2105.09259/) (2021)
- [Conditional Variational Autoencoder for Sign Language Translation with Cross-Modal Alignment](https://mlsanity.com/paper/2312.15645/) (2023)
- [Modeling Bilingual Conversational Characteristics for Neural Chat Translation](https://mlsanity.com/paper/2107.11164/) (2021)
- [A Variational Hierarchical Model for Neural Cross-Lingual Summarization](https://mlsanity.com/paper/2203.03820/) (2022)
- [Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework](https://mlsanity.com/paper/2203.09053/) (2022)
- [WR-ONE2SET: Towards Well-Calibrated Keyphrase Generation](https://mlsanity.com/paper/2211.06862/) (2022)
- [Revisiting Bayesian Autoencoders with MCMC](https://mlsanity.com/paper/2104.05915/) (2021)
- [Bayesian graph convolutional neural networks via tempered MCMC](https://mlsanity.com/paper/2104.08438/) (2021)
- [Confidence Based Bidirectional Global Context Aware Training Framework for Neural Machine Translation](https://mlsanity.com/paper/2202.13663/) (2022)
- [SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers](https://mlsanity.com/paper/2209.06442/) (2022)
- [Math Word Problem Solving by Generating Linguistic Variants of Problem Statements](https://mlsanity.com/paper/2306.13899/) (2023)
- [KGR^4: Retrieval, Retrospect, Refine and Rethink for Commonsense Generation](https://mlsanity.com/paper/2112.08266/) (2021)
- [INTERACTION: A Generative XAI Framework for Natural Language Inference Explanations](https://mlsanity.com/paper/2209.01061/) (2022)
- [Conversational Speech Recognition by Learning Audio-textual Cross-modal Contextual Representation](https://mlsanity.com/paper/2310.14278/) (2023)
- [A Joint Learning Model with Variational Interaction for Multilingual Program Translation](https://mlsanity.com/paper/2408.14515/) (2024)
- [Discrete Auto-regressive Variational Attention Models for Text Modeling](https://mlsanity.com/paper/2106.08571/) (2021)
- [HireVAE: An Online and Adaptive Factor Model Based on Hierarchical and Regime-Switch VAE](https://mlsanity.com/paper/2306.02848/) (2023)
- [Deep Learning Models in Software Requirements Engineering](https://mlsanity.com/paper/2105.07771/) (2021)
- [Turning Flowchart into Dialog: Augmenting Flowchart-grounded Troubleshooting Dialogs via Synthetic Data Generation](https://mlsanity.com/paper/2305.01323/) (2023)
- [Shared Latent Space by Both Languages in Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2305.03511/) (2023)
- [Learning low-dimensional dynamics from whole-brain data improves task capture](https://mlsanity.com/paper/2305.14369/) (2023)
- [Boosting Summarization with Normalizing Flows and Aggressive Training](https://mlsanity.com/paper/2311.00588/) (2023)

## Abstract (arXiv)

Partially inspired by successful applications of variational recurrent neural networks, we propose a novel variational recurrent neural machine translation (VRNMT) model in this paper. Different from the variational NMT, VRNMT introduces a series of latent random variables to model the translation procedure of a sentence in a generative way, instead of a single latent variable. Specifically, the latent random variables are included into the hidden states of the NMT decoder with elements from the variational autoencoder. In this way, these variables are recurrently generated, which enables them to further capture strong and complex dependencies among the output translations at different timesteps. In order to deal with the challenges in performing efficient posterior inference and large-scale training during the incorporation of latent variables, we build a neural posterior approximator, and equip it with a reparameterization technique to estimate the variational lower bound. Experiments on Chinese-English and English-German translation tasks demonstrate that the proposed model achieves significant improvements over both the conventional and variational NMT models.