mlsanityData to 13 Oct 2026

Revisiting the Markov Property for Machine Translation

arXiv 2402.02084 ↗First posted 3 Feb 2024cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper, we re-examine the Markov property in the context of neural machine translation. We design a Markov Autoregressive Transformer~(MAT) and undertake a comprehensive assessment of its performance across four WMT benchmarks. Our findings indicate that MAT with an order larger than 4 can generate translations with quality on par with that of conventional autoregressive transformers. In addition, counter-intuitively, we also find that the advantages of utilizing a higher-order MAT do not specifically contribute to the translation of longer sentences.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 18 of its 32 references that are in the map. First posted on arXiv 3 Feb 2024.

Builds on

18 of 32 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,106
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,539
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,981
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,910
  5. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,810
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,793
  7. 2017Convolutional Sequence to Sequence Learningcited by 552
  8. 2017Non-Autoregressive Neural Machine Translationcited by 462
  9. 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
  10. 2019Pay Less Attention with Lightweight and Dynamic Convolutionscited by 211
  11. 2021Scaling Laws for Neural Machine Translationcited by 111
  12. 2023Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Methodcited by 97
6 more
  1. 2019Fast Structured Decoding for Sequence Modelscited by 59
  2. 2021Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 57
  3. 2020Efficient Second-Order TreeCRF for Neural Dependency Parsingcited by 48
  4. 2017Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRFcited by 16
  5. 2020Cascaded Text Generation with Markov Transformerscited by 7
  6. 2020Faster Transformer Decoding: N-gram Masked Self-Attentioncited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMscited by 96
  2. 2025Sparse-to-Dense: A Free Lunch for Lossless Acceleration of Video Understanding in LLMscited by 3