mlsanityData to 14 Oct 2026

Convolutional Sequence to Sequence Learning

arXiv 1705.03122 ↗First posted 8 May 2017cs.CLcited by 552

Not on the map. Why?

Abstract (arXiv)

The prevalent approach to sequence to sequence learning maps an input sequence to a variable length output sequence via recurrent neural networks. We introduce an architecture based entirely on convolutional neural networks. Compared to recurrent models, computations over all elements can be fully parallelized during training and optimization is easier since the number of non-linearities is fixed and independent of the input length. Our use of gated linear units eases gradient propagation and we equip each decoder layer with a separate attention module. We outperform the accuracy of the deep LSTM setup of Wu et al. (2016) on both WMT'14 English-German and WMT'14 English-French translation at an order of magnitude faster speed, both on GPU and CPU.

Key facts. Cited by 552 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 0; 2021: 173; 2022: 140; 2023: 86; 2024: 74; 2025: 59; 2026: 18.

552 papers.

Sort:

By area:

  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  3. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  4. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  5. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  6. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  7. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  8. 2022Transformers in Time Series: A Surveycited by 497
  9. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  10. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
  11. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 406
  12. 2021A Survey of Transformerscited by 401
38 more
  1. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 347
  2. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 339
  3. 2022Behavior Transformers: Cloning k modes with one stonecited by 304
  4. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  5. 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 217
  6. 2021Rethinking and Improving Relative Position Encoding for Vision Transformercited by 207
  7. 2022Transformer Quality in Linear Timecited by 191
  8. 2022Mega: Moving Average Equipped Gated Attentioncited by 179
  9. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 150
  10. 2022SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fieldscited by 144
  11. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  12. 2022MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generationcited by 125
  13. 2022A Survey of Deep Learning for Mathematical Reasoningcited by 124
  14. 2023Randomized Positional Encodings Boost Length Generalization of Transformerscited by 117
  15. 2022EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformerscited by 116
  16. 2022Overcoming a Theoretical Limitation of Self-Attentioncited by 113
  17. 2021Fully Hyperbolic Neural Networkscited by 108
  18. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  19. 2021Position Information in Transformers: An Overviewcited by 103
  20. 2021Finetuning Pretrained Transformers into RNNscited by 96
  21. 2021Pretrained Language Models for Text Generation: A Surveycited by 95
  22. 2021Learnable Fourier Features for Multi-Dimensional Spatial Positional Encodingcited by 93
  23. 2023Protecting Language Generation Models via Invisible Watermarkingcited by 90
  24. 2021CURE: Code-Aware Neural Machine Translation for Automatic Program Repaircited by 86
  25. 2021Gradients are Not All You Needcited by 86
  26. 2022KERPLE: Kernelized Relative Positional Embedding for Length Extrapolationcited by 86
  27. 2022Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learningcited by 86
  28. 2021Space-Time-Separable Graph Convolutional Network for Pose Forecastingcited by 84
  29. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  30. 2022DFNet: Enhance Absolute Pose Regression with Direct Feature Matchingcited by 82
  31. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarkscited by 76
  32. 2025Test-time regression: a unifying framework for designing sequence models with associative memorycited by 74
  33. 2021Aligning Latent and Image Spaces to Connect the Unconnectablecited by 70
  34. 2021Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Futurecited by 67
  35. 2024Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Maskingcited by 65
  36. 2023PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networkscited by 64
  37. 2021Anomaly Detection in Dynamic Graphs via Transformercited by 59
  38. 2021Initialization and Regularization of Factorized Neural Layerscited by 58

50 most cited of 552 All 552 (JSON)