mlsanityData to 14 Oct 2026

MobileNMT: Enabling Translation in 15MB and 30ms

arXiv 2306.04235 ↗First posted 7 Jun 2023cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Deploying NMT models on mobile devices is essential for privacy, low latency, and offline scenarios. For high model capacity, NMT models are rather large. Running these models on devices is challenging with limited storage, memory, computation, and power consumption. Existing work either only focuses on a single metric such as FLOPs or general engine which is not good at auto-regressive decoding. In this paper, we present MobileNMT, a system that can translate in 15MB and 30ms on devices. We propose a series of principles for model compression when combined with quantization. Further, we implement an engine that is friendly to INT8 and decoding. With the co-design of model and engine, compared with the existing system, we speed up 47.0x and save 99.5% of memory with only 11.6% loss of BLEU. The code is publicly available at https://github.com/zjersey/Lightseq-ARM.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 41 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

35 of 41 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2016Layer Normalizationcited by 5,669
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  9. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  10. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  11. 2017Mixed Precision Trainingcited by 1,156
  12. 2020On Layer Normalization in the Transformer Architecturecited by 922
23 more
  1. 2018Universal Transformerscited by 647
  2. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  3. 2019Learning Deep Transformer Models for Machine Translationcited by 309
  4. 2018Adaptive Input Representations for Neural Language Modelingcited by 226
  5. 2018Scaling Neural Machine Translationcited by 204
  6. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 193
  7. 2022DeepNet: Scaling Transformers to 1,000 Layerscited by 178
  8. 2019Transformers without Tears: Improving the Normalization of Self-Attentioncited by 153
  9. 2020Lite Transformer with Long-Short Range Attentioncited by 127
  10. 2017Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracycited by 101
  11. 2021Network Quantization with Element-wise Gradient Scalingcited by 87
  12. 2021Lessons on Parameter Sharing across Layers in Transformerscited by 85
  13. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 79
  14. 2016Efficient softmax approximation for GPUscited by 67
  15. 2020Vocabulary Learning via Optimal Transport for Neural Machine Translationcited by 58
  16. 2020Very Deep Transformers for Neural Machine Translationcited by 50
  17. 2019QKD: Quantization-aware Knowledge Distillationcited by 44
  18. 2015Resiliency of Deep Neural Networks under Quantizationcited by 35
  19. 2019A Call for Prudent Choice of Subword Merge Operations in Neural Machine Translationcited by 30
  20. 2018SYQ: Learning Symmetric Quantization For Efficient Deep Neural Networkscited by 24
  21. 2022EdgeFormer: A Parameter-Efficient Transformer for On-Device Seq2seq Generationcited by 21
  22. 2018Pieces of Eight: 8-bit Neural Machine Translationcited by 7
  23. 2021Bag of Tricks for Optimizing Transformer Efficiencycited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026LatentMT: Machine Translation with Latent Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024On the Compressibility of Quantized Large Language Models
  2. 2025Optimizing LLMs Using Quantization for Mobile Execution
  3. 2024MobileQuant: Mobile-friendly Quantization for On-device Language Models
  4. 2024PalmBench: A Comprehensive Benchmark of Compressed Large Language Models on Mobile Platforms
  5. 2025Scaling LLM Test-Time Compute with Mobile NPU on Smartphones
  6. 2025MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices
  7. 2023LinguaLinked: A Distributed Large Language Model Inference System for Mobile Devices
  8. 2024On-Device Language Models: A Comprehensive Review