mlsanityData to 14 Oct 2026

Context-Aware Cross-Attention for Non-Autoregressive Translation

arXiv 2011.00770 ↗First posted 2 Nov 2020cs.CLcited by 26

Not on the map. Why?

Abstract (arXiv)

Non-autoregressive translation (NAT) significantly accelerates the inference process by predicting the entire target sequence. However, due to the lack of target dependency modelling in the decoder, the conditional generation process heavily depends on the cross-attention. In this paper, we reveal a localness perception problem in NAT cross-attention, for which it is difficult to adequately capture source context. To alleviate this problem, we propose to enhance signals of neighbour source tokens into conventional cross-attention. Experimental results on several representative datasets show that our approach can consistently improve translation quality over strong NAT baselines. Extensive analyses demonstrate that the enhanced cross-attention achieves better exploitation of source contexts by leveraging both local and global information.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 9; 2022: 10; 2023: 5; 2024: 1; 2025: 1.

26 papers.

Sort:

By area:

  1. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  2. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49
  3. 2021Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translationcited by 34
  4. 2022Learning Graph Neural Networks for Image Style Transfercited by 29
  5. 2021Progressive Multi-Granularity Training for Non-Autoregressive Translationcited by 25
  6. 2023Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?cited by 21
  7. 2022E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generationcited by 19
  8. 2021Modeling Concentrated Cross-Attention for Neural Machine Translation with Gaussian Mixture Modelcited by 18
  9. 2021Improving Neural Machine Translation by Bidirectional Trainingcited by 17
  10. 2022Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-based Sentiment Analysiscited by 16
  11. 2022On the Complementarity between Pre-Training and Random-Initialization for Resource-Rich Machine Translationcited by 15
  12. 2021The USYD-JD Speech Translation System for IWSLT 2021cited by 11
14 more
  1. 2023Revisiting Token Dropping Strategy in Efficient BERT Pretrainingcited by 11
  2. 2021Bridging the Gap Between Clean Data Training and Real-World Inference for Spoken Language Understandingcited by 9
  3. 2021Self-Guided Curriculum Learning for Neural Machine Translationcited by 9
  4. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  5. 2022Vega-MT: The JD Explore Academy Translation System for WMT22cited by 9
  6. 2022BLISS: Robust Sequence-to-Sequence Learning via Self-Supervised Input Representationcited by 8
  7. 2022Non-Autoregressive Neural Machine Translation: A Call for Claritycited by 5
  8. 2021MirrorAlign: A Super Lightweight Unsupervised Word Alignment Model via Cross-Lingual Contrastive Learningcited by 4
  9. 2023Prompt-Learning for Cross-Lingual Relation Extractioncited by 4
  10. 2025Curriculum Learning for Biological Sequence Prediction: The Case of De Novo Peptide Sequencingcited by 4
  11. 2022Original or Translated? On the Use of Parallel Data for Translation Quality Estimationcited by 2
  12. 2023OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML Systemcited by 2
  13. 2023Revisiting Non-Autoregressive Translation at Scalecited by 2
  14. 2022Tri-Attention: Explicit Context-Aware Attention Mechanism for Natural Language Processingcited by 1