mlsanityData to 7 Oct 2026

TranSFormer: Slow-Fast Transformer for Machine Translation

arXiv 2305.16982 ↗First posted 26 May 2023cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Learning multiscale Transformer models has been evidenced as a viable approach to augmenting machine translation systems. Prior research has primarily focused on treating subwords as basic units in developing such systems. However, the incorporation of fine-grained character-level features into multiscale Transformer has not yet been explored. In this work, we present a Slow-Fast two-stream learning model, referred to as TranSFormer, which utilizes a ``slow'' branch to deal with subword sequences and a ``fast'' branch to deal with longer character sequences. This model is efficient since the fast branch is very lightweight by reducing the model width, and yet provides useful fine-grained features for the slow branch. Our TranSFormer shows consistent BLEU improvements (larger than 1 BLEU point) on several machine translation benchmarks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 31 of its 40 references that are in the map. First posted on arXiv 26 May 2023.

Builds on

31 of 40 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2018SlowFast Networks for Video Recognitioncited by 1,862
  7. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  9. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  10. 2018Self-Attention with Relative Position Representationscited by 1,112
  11. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  12. 2021Transformer in Transformercited by 632
19 more
  1. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  2. 2018Scaling Neural Machine Translationcited by 204
  3. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  4. 2018Achieving Human Parity on Automatic Chinese to English News Translationcited by 119
  5. 2018Context-Aware Neural Machine Translation Learns Anaphora Resolutioncited by 97
  6. 2016Fully Character-Level Neural Machine Translation without Explicit Segmentationcited by 55
  7. 2018Modeling Localness for Self-Attention Networkscited by 43
  8. 2019Convolutional Self-Attention Networkscited by 32
  9. 2018Revisiting Character-Based Neural Machine Translation with Capacity and Compressioncited by 28
  10. 2022ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generationcited by 26
  11. 2019MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learningcited by 19
  12. 2020DeLighT: Deep and Light-weight Transformercited by 17
  13. 2017Weighted Transformer Network for Machine Translationcited by 16
  14. 2019Multi-Scale Self-Attention for Text Classificationcited by 15
  15. 2020Multi-Unit Transformers for Neural Machine Translationcited by 13
  16. 2020Character-Level Translation with Self-attentioncited by 12
  17. 2019Multi-Granularity Self-Attention for Neural Machine Translationcited by 10
  18. 2020Multi-branch Attentive Transformercited by 9
  19. 2022Learning Multiscale Transformer Models for Sequence Generationcited by 7

Built on by

Cited by year

Citing papers per year: 2023: 4.

4 papers.

  1. 2023VIDiff: Translating Videos via Multi-Modal Instructions with Diffusion Modelscited by 19
  2. 2023PanoSwin: a Pano-style Swin Transformer for Panorama Understandingcited by 18
  3. 2023Core Building Blocks: Next Gen Geo Spatial GPT Applicationcited by 4
  4. 2023Enhancing Neural Machine Translation with Semantic Unitscited by 2