# A Convolutional Encoder Model for Neural Machine Translation

arXiv: https://arxiv.org/abs/1611.02344 · HTML: https://mlsanity.com/paper/1611.02344/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2016.

## Built on by (most cited first, 30 of 54)

- [Adaptive Graph Convolution for Point Cloud Analysis](https://mlsanity.com/paper/2108.08035/) (2021)
- [Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolution](https://mlsanity.com/paper/2204.13062/) (2022)
- [Deep Learning Approaches on Image Captioning: A Review](https://mlsanity.com/paper/2201.12944/) (2022)
- [Grammatical Error Correction: A Survey of the State of the Art](https://mlsanity.com/paper/2211.05166/) (2022)
- [Simplicial Attention Networks](https://mlsanity.com/paper/2204.09455/) (2022)
- [Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networks](https://mlsanity.com/paper/2103.04886/) (2021)
- [Deep networks for system identification: a Survey](https://mlsanity.com/paper/2301.12832/) (2023)
- [Video-Mined Task Graphs for Keystep Recognition in Instructional Videos](https://mlsanity.com/paper/2307.08763/) (2023)
- [Generalized Simplicial Attention Neural Networks](https://mlsanity.com/paper/2309.02138/) (2023)
- [Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directions](https://mlsanity.com/paper/2307.07527/) (2023)
- [Simplicial Attention Neural Networks](https://mlsanity.com/paper/2203.07485/) (2022)
- [Improving End-to-End Text Image Translation From the Auxiliary Text Translation Task](https://mlsanity.com/paper/2210.03887/) (2022)
- [How well do you know your summarization datasets?](https://mlsanity.com/paper/2106.11388/) (2021)
- [A Survey on Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2107.04239/) (2021)
- [DeepTrack: Lightweight Deep Learning for Vehicle Path Prediction in Highways](https://mlsanity.com/paper/2108.00505/) (2021)
- [Universal Multimodal Representation for Language Understanding](https://mlsanity.com/paper/2301.03344/) (2023)
- [Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks](https://mlsanity.com/paper/2201.02143/) (2022)
- [AGConv: Adaptive Graph Convolution on 3D Point Clouds](https://mlsanity.com/paper/2206.04665/) (2022)
- [Distill the Image to Nowhere: Inversion Knowledge Distillation for Multimodal Machine Translation](https://mlsanity.com/paper/2210.04468/) (2022)
- [Neural OCR Post-Hoc Correction of Historical Corpora](https://mlsanity.com/paper/2102.00583/) (2021)
- [Parallel Spatio-Temporal Attention-Based TCN for Multivariate Time Series Prediction](https://mlsanity.com/paper/2203.00971/) (2022)
- [Towards Better Chinese-centric Neural Machine Translation for Low-resource Languages](https://mlsanity.com/paper/2204.04344/) (2022)
- [GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation](https://mlsanity.com/paper/2207.14467/) (2022)
- [diffConv: Analyzing Irregular Point Clouds with an Irregular View](https://mlsanity.com/paper/2111.14658/) (2021)
- [Intensity-free Convolutional Temporal Point Process: Incorporating Local and Global Event Contexts](https://mlsanity.com/paper/2306.14072/) (2023)
- [Graph Language Models](https://mlsanity.com/paper/2401.07105/) (2024)
- [The Effect of Normalization for Bi-directional Amharic-English Neural Machine Translation](https://mlsanity.com/paper/2210.15224/) (2022)
- [MultiMax: Sparse and Multi-Modal Attention Learning](https://mlsanity.com/paper/2406.01189/) (2024)
- [Adversarially Regularized Graph Attention Networks for Inductive Learning on Partially Labeled Graphs](https://mlsanity.com/paper/2106.03393/) (2021)
- [Joint Generator-Ranker Learning for Natural Language Generation](https://mlsanity.com/paper/2206.13974/) (2022)

## Abstract (arXiv)

The prevalent approach to neural machine translation relies on bi-directional LSTMs to encode the source sentence. In this paper we present a faster and simpler architecture based on a succession of convolutional layers. This allows to encode the entire source sentence simultaneously compared to recurrent networks for which computation is constrained by temporal dependencies. On WMT'16 English-Romanian translation we achieve competitive accuracy to the state-of-the-art and we outperform several recently published results on the WMT'15 English-German task. Our models obtain almost the same accuracy as a very deep LSTM setup on WMT'14 English-French translation. Our convolutional encoder speeds up CPU decoding by more than two times at the same or higher accuracy as a strong bi-directional LSTM baseline.