mlsanityData to 7 Oct 2026

Enhancing Neural Machine Translation with Semantic Units

arXiv 2310.11360 ↗First posted 17 Oct 2023cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Conventional neural machine translation (NMT) models typically use subwords and words as the basic units for model input and comprehension. However, complete words and phrases composed of several tokens are often the fundamental units for expressing semantics, referred to as semantic units. To address this issue, we propose a method Semantic Units for Machine Translation (SU4MT) which models the integral meanings of semantic units within a sentence, and then leverages them to provide a new perspective for understanding the sentence. Specifically, we first propose Word Pair Encoding (WPE), a phrase extraction method to help identify the boundaries of semantic units. Next, we design an Attentive Semantic Fusion (ASF) layer to integrate the semantics of multiple subwords into a single vector: the semantic unit representation. Lastly, the semantic-unit-level sentence representation is concatenated to the token-level one, and they are combined as the input of encoder. Experimental results demonstrate that our method effectively models and leverages semantic-unit-level information and outperforms the strong baselines. The code is available at https://github.com/ictnlp/SU4MT.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 25 of its 46 references that are in the map. First posted on arXiv 17 Oct 2023.

Builds on

25 of 46 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  9. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  10. 2017Convolutional Sequence to Sequence Learningcited by 550
  11. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  12. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
13 more
  1. 2018Phrase-Based & Neural Unsupervised Machine Translationcited by 138
  2. 2021Learning Compact Metrics for MTcited by 93
  3. 2017Graph Convolutional Encoders for Syntax-aware Neural Machine Translationcited by 70
  4. 2023BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Modelscited by 42
  5. 2022UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Miningcited by 33
  6. 2016Tree-to-Sequence Attentional Neural Machine Translationcited by 28
  7. 2022Neural Machine Translation with Phrase-Level Universal Visual Representationscited by 26
  8. 2019Multi-Granularity Self-Attention for Neural Machine Translationcited by 10
  9. 2022Learning Multiscale Transformer Models for Sequence Generationcited by 7
  10. 2021Semantics-aware Attention Improves Neural Machine Translationcited by 5
  11. 2020Learning Source Phrase Representations for Neural Machine Translationcited by 4
  12. 2023TranSFormer: Slow-Fast Transformer for Machine Translationcited by 4
  13. 2022Categorizing Semantic Representations for Neural Machine Translationcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024Integrating Multi-scale Contextualized Information for Byte-based Neural Machine Translationcited by 1
  2. 2024MoCE: Adaptive Mixture of Contextualization Experts for Byte-based Neural Machine Translationcited by 0