# Neural Machine Translation via Binary Code Prediction

arXiv: https://arxiv.org/abs/1704.06918 · HTML: https://mlsanity.com/paper/1704.06918/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2017.

## Built on by (most cited first, 2 of 2)

- [Recurrent Stacking of Layers in Neural Networks: An Application to Neural Machine Translation](https://mlsanity.com/paper/2106.10002/) (2021)
- [BitLM: Unlocking Multi-Token Language Generation with Bitwise Continuous Diffusion](https://mlsanity.com/paper/2605.11577/) (2026)

## Abstract (arXiv)

In this paper, we propose a new method for calculating the output layer in neural machine translation systems. The method is based on predicting a binary code for each word and can reduce computation time/memory requirements of the output layer to be logarithmic in vocabulary size in the best case. In addition, we also introduce two advanced approaches to improve the robustness of the proposed model: using error-correcting codes and combining softmax and binary codes. Experiments on two English-Japanese bidirectional translation tasks show proposed models achieve BLEU scores that approach the softmax, while reducing memory usage to the order of less than 1/10 and improving decoding speed on CPUs by x5 to x10.