mlsanityData to 7 Oct 2026

Fully Quantized Transformer for Machine Translation

arXiv 1910.10485 ↗First posted 17 Oct 2019cs.CL cs.LG stat.MLcited by 36

Not on the map. Why?

Abstract (arXiv)

State-of-the-art neural machine translation methods employ massive amounts of parameters. Drastically reducing computational costs of such methods without affecting performance has been up to this point unsuccessful. To this end, we propose FullyQT: an all-inclusive quantization strategy for the Transformer. To the best of our knowledge, we are the first to show that it is possible to avoid any loss in translation quality with a fully quantized Transformer. Indeed, compared to full-precision, our 8-bit models score greater or equal BLEU on most tasks. Comparing ourselves to all previously proposed methods, we achieve state-of-the-art quantization results.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 10; 2023: 10; 2024: 2; 2025: 2; 2026: 1.

36 papers.

Sort:

By area:

  1. 2022DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptationcited by 229
  2. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  3. 2021Sparse is Enough in Scaling Transformerscited by 86
  4. 2021Scalable Vision Transformers with Hierarchical Poolingcited by 84
  5. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  6. 2021A Survey on Green Deep Learningcited by 52
  7. 2022HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformerscited by 46
  8. 2022A Survey on Model Compression and Acceleration for Pretrained Language Modelscited by 45
  9. 2023The Closeness of In-Context Learning and Weight Shifting for Softmax Regressioncited by 41
  10. 2022SimA: Simple Softmax-free Attention for Vision Transformerscited by 32
  11. 2021Kronecker Decomposition for GPT Compressioncited by 27
  12. 2022DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantizationcited by 23
24 more
  1. 2021KroneckerBERT: Learning Kronecker Decomposition for Pre-trained Language Models via Knowledge Distillationcited by 19
  2. 2021RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillationcited by 18
  3. 2023SwiftTron: An Efficient Hardware Accelerator for Quantized Transformerscited by 17
  4. 2021StreamHover: Livestream Transcript Summarization and Annotationcited by 15
  5. 2021On the Distribution, Sparsity, and Inference-time Quantization of Attention Values in Transformerscited by 14
  6. 2023Binarized Neural Machine Translationcited by 12
  7. 2022Quadapter: Adapter for GPT-2 Quantizationcited by 10
  8. 2022Approximate Computing and the Efficient Machine Learning Expeditioncited by 8
  9. 2023Training Acceleration of Low-Rank Decomposed Networks using Sequential Freezing and Rank Quantizationcited by 8
  10. 2023Speeding up Resnet Architecture with Layers Targeted Low Rank Decompositioncited by 8
  11. 2023Binary and Ternary Natural Language Generationcited by 7
  12. 2023Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inferencecited by 7
  13. 2021Sampling Equivariant Self-attention Networks for Object Detection in Aerial Imagescited by 6
  14. 2022ACORT: A Compact Object Relation Transformer for Parameter Efficient Image Captioningcited by 6
  15. 2025AnchorFormer: Differentiable Anchor Attention for Efficient Vision Transformercited by 3
  16. 2022The Ecological Footprint of Neural Machine Translation Systemscited by 2
  17. 2023Self-Distilled Quantization: Achieving High Compression Rates in Transformer-Based Language Modelscited by 2
  18. 2023Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Modelscited by 2
  19. 2021Dynamic Multi-Branch Layers for On-Device Neural Machine Translationcited by 1
  20. 2023YFlows: Systematic Dataflow Exploration and Code Generation for Efficient Neural Network Inference using SIMD Architectures on CPUscited by 1
  21. 2024Quantized Approximately Orthogonal Recurrent Neural Networkscited by 1
  22. 2024Accelerating the Low-Rank Decomposed Modelscited by 1
  23. 2025HILGEN: Hierarchically-Informed Data Generation for Biomedical NER Using Knowledgebases and Large Language Modelscited by 0
  24. 2026Efficient Multilingual Neural Machine Translation via Corpus-Driven Vocabulary Pruning: An English-Arabic Case Studycited by 0