mlsanityData to 7 Oct 2026

Neural Machine Translation by Jointly Learning to Align and Translate

arXiv 1409.0473 ↗First posted 1 Sep 2014cs.CL cs.LG cs.NEcited by 4,530

Not on the map. Why?

Cited by 45 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural machine translation often belong to a family of encoder-decoders and consists of an encoder that encodes a source sentence into a fixed-length vector from which a decoder generates a translation. In this paper, we conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and propose to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. With this new approach, we achieve a translation performance comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation. Furthermore, qualitative analysis reveals that the (soft-)alignments found by the model agree well with our intuition.

Key facts. Cited by 4,530 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Sep 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 2; 2018: 9; 2019: 2; 2020: 9; 2021: 1348; 2022: 964; 2023: 808; 2024: 655; 2025: 472; 2026: 261.

4,530 papers.

Sort:

By area:

  1. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2023A Survey of Large Language Modelscited by 2,537
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  7. 2021Vision Transformers for Dense Predictioncited by 1,789
  8. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  9. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  10. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  11. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  12. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
38 more
  1. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  2. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  3. 2021Multiscale Vision Transformerscited by 896
  4. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  5. 2021How Attentive are Graph Attention Networks?cited by 818
  6. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  7. 2023A Comprehensive Overview of Large Language Modelscited by 639
  8. 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 549
  9. 2023The Falcon Series of Open Language Modelscited by 526
  10. 2023Zero-shot Image-to-Image Translationcited by 503
  11. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  12. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  13. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  14. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  15. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  16. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  17. 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 411
  18. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  19. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 373
  20. 2021Pay Attention to MLPscited by 367
  21. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  22. 2021Random Feature Attentioncited by 350
  23. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  24. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 345
  25. 2021FNet: Mixing Tokens with Fourier Transformscited by 343
  26. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  27. 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 335
  28. 2021Choose a Transformer: Fourier or Galerkincited by 333
  29. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 327
  30. 2024Titans: Learning to Memorize at Test Timecited by 303
  31. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  32. 2021GFlowNet Foundationscited by 286
  33. 2023A Long Way to Go: Investigating Length Correlations in RLHFcited by 285
  34. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  35. 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
  36. 2021TransBTS: Multimodal Brain Tumor Segmentation Using Transformercited by 265
  37. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  38. 2021GRAND: Graph Neural Diffusioncited by 258

50 most cited of 4,530 All 4,530 (JSON)