mlsanityData to 7 Oct 2026

Effective Approaches to Attention-based Neural Machine Translation

arXiv 1508.04025 ↗First posted 17 Aug 2015cs.CLcited by 1,024

Not on the map. Why?

Abstract (arXiv)

An attentional mechanism has lately been used to improve neural machine translation (NMT) by selectively focusing on parts of the source sentence during translation. However, there has been little work exploring useful architectures for attention-based NMT. This paper examines two simple and effective classes of attentional mechanism: a global approach which always attends to all source words and a local one that only looks at a subset of source words at a time. We demonstrate the effectiveness of both approaches over the WMT translation tasks between English and German in both directions. With local attention, we achieve a significant gain of 5.0 BLEU points over non-attentional systems which already incorporate known techniques such as dropout. Our ensemble model using different attention architectures has established a new state-of-the-art result in the WMT'15 English to German translation task with 25.9 BLEU points, an improvement of 1.0 BLEU points over the existing best system backed by NMT and an n-gram reranker.

Key facts. Cited by 1,024 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Aug 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 2; 2018: 2; 2019: 2; 2020: 3; 2021: 384; 2022: 246; 2023: 152; 2024: 123; 2025: 75; 2026: 35.

1,024 papers.

Sort:

By area:

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  2. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  3. 2025Gemma 3 Technical Reportcited by 1,686
  4. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  5. 2021How Attentive are Graph Attention Networks?cited by 818
  6. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  7. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  8. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  9. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  10. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 327
  11. 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
  12. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
38 more
  1. 2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 251
  2. 2022A Contrastive Framework for Neural Text Generationcited by 224
  3. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  4. 2021Thinking Like Transformerscited by 182
  5. 2022TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentialscited by 179
  6. 2022Mega: Moving Average Equipped Gated Attentioncited by 178
  7. 2021Improving Sign Language Translation with Monolingual Data by Sign Back-Translationcited by 177
  8. 2023Implicit Chain of Thought Reasoning via Knowledge Distillationcited by 165
  9. 2021Retrieval Augmented Code Generation and Summarizationcited by 148
  10. 2021Inductive Biases and Variable Creation in Self-Attention Mechanismscited by 144
  11. 2021A Unified Generative Framework for Various NER Subtaskscited by 141
  12. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
  13. 2023Scalable Transformer for PDE Surrogate Modelingcited by 123
  14. 2021PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Modelscited by 118
  15. 2021DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformercited by 111
  16. 2021Recent advances and clinical applications of deep learning in medical image analysiscited by 104
  17. 2023Gloss-free Sign Language Translation: Improving from Visual-Language Pretrainingcited by 104
  18. 2022How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervisioncited by 102
  19. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  20. 2021Unsupervised Discovery of Object Radiance Fieldscited by 100
  21. 2021A Unified Generative Framework for Aspect-Based Sentiment Analysiscited by 97
  22. 2021From Two to One: A New Scene Text Recognizer with Visual Language Modeling Networkcited by 97
  23. 2024Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilizationcited by 97
  24. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 95
  25. 2021Learnable Fourier Features for Multi-Dimensional Spatial Positional Encodingcited by 93
  26. 2021Do Long-Range Language Models Actually Use Long-Range Context?cited by 92
  27. 2022Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexitycited by 92
  28. 2021X-FACT: A New Benchmark Dataset for Multilingual Fact Checkingcited by 90
  29. 2022How Powerful are K-hop Message Passing Graph Neural Networkscited by 90
  30. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
  31. 2023Collaborative Perception in Autonomous Driving: Methods, Datasets and Challengescited by 87
  32. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  33. 2022MapQA: A Dataset for Question Answering on Choropleth Mapscited by 82
  34. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  35. 2021SciCap: Generating Captions for Scientific Figurescited by 76
  36. 2024LLMs are Good Sign Language Translatorscited by 76
  37. 2021Is Attention Better Than Matrix Decomposition?cited by 75
  38. 2023Exposing Attention Glitches with Flip-Flop Language Modelingcited by 74

50 most cited of 1,024 All 1,024 (JSON)