# Leveraging Automated Unit Tests for Unsupervised Code Translation

arXiv: https://arxiv.org/abs/2110.06773 · HTML: https://mlsanity.com/paper/2110.06773/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 62 references that are in the map.
- First posted on arXiv 13 Oct 2021.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [Unified Pre-training for Program Understanding and Generation](https://mlsanity.com/paper/2103.06333/) (2021)
- [Synthetic and Natural Noise Both Break Neural Machine Translation](https://mlsanity.com/paper/1711.02173/) (2017)
- [Unsupervised Translation of Programming Languages](https://mlsanity.com/paper/2006.03511/) (2020)
- [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017)
- [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019)
- [SPoC: Search-based Pseudocode to Code](https://mlsanity.com/paper/1906.04908/) (2019)
- [SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair](https://mlsanity.com/paper/1901.01808/) (2018)
- [Phrase-Based & Neural Unsupervised Machine Translation](https://mlsanity.com/paper/1804.07755/) (2018)
- [Unit Test Case Generation with Transformers and Focal Context](https://mlsanity.com/paper/2009.05617/) (2020)
- [An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation](https://mlsanity.com/paper/1812.08693/) (2018)
- [Graph-based, Self-Supervised Program Repair from Diagnostic Feedback](https://mlsanity.com/paper/2005.10636/) (2020)
- [Tree-to-tree Neural Networks for Program Translation](https://mlsanity.com/paper/1802.03691/) (2018)
- [On the Impact of Various Types of Noise on Neural Machine Translation](https://mlsanity.com/paper/1805.12282/) (2018)
- [Code Prediction by Feeding Trees to Transformers](https://mlsanity.com/paper/2003.13848/) (2020)
- [DOBF: A Deobfuscation Pre-Training Objective for Programming Languages](https://mlsanity.com/paper/2102.07492/) (2021)
- [A parallel corpus of Python functions and documentation strings for automated code documentation and code generation](https://mlsanity.com/paper/1707.02275/) (2017)
- [SlimIPL: Language-Model-Free Iterative Pseudo-Labeling](https://mlsanity.com/paper/2010.11524/) (2020)
- [Code Completion with Neural Attention and Pointer Networks](https://mlsanity.com/paper/1711.09573/) (2017)
- [Fast and Memory-Efficient Neural Code Completion](https://mlsanity.com/paper/2004.13651/) (2020)
- [Learning to Fix Build Errors with Graph2Diff Neural Networks](https://mlsanity.com/paper/1911.01205/) (2019)

## Built on by (most cited first, 30 of 82)

- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [Agentless: Demystifying LLM-based Software Engineering Agents](https://mlsanity.com/paper/2407.01489/) (2024)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [Planning with Large Language Models for Code Generation](https://mlsanity.com/paper/2303.05510/) (2023)
- [Language Models Can Teach Themselves to Program Better](https://mlsanity.com/paper/2207.14502/) (2022)
- [CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation](https://mlsanity.com/paper/2310.04951/) (2023)
- [Exploring and Unleashing the Power of Large Language Models in Automated Code Translation](https://mlsanity.com/paper/2404.14646/) (2024)
- [xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval](https://mlsanity.com/paper/2303.03004/) (2023)
- [Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMs](https://mlsanity.com/paper/2308.09895/) (2023)
- [Code Translation with Compiler Representations](https://mlsanity.com/paper/2207.03578/) (2022)
- [CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation](https://mlsanity.com/paper/2501.07811/) (2025)
- [VERINA: Benchmarking Verifiable Code Generation](https://mlsanity.com/paper/2505.23135/) (2025)
- [CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution](https://mlsanity.com/paper/2408.13001/) (2024)
- [AlphaTrans: A Neuro-Symbolic Compositional Approach for Repository-Level Code Translation and Validation](https://mlsanity.com/paper/2410.24117/) (2024)
- [IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generators](https://mlsanity.com/paper/2403.03894/) (2024)
- [CoTran: An LLM-based Code Translator using Reinforcement Learning with Feedback from Compiler and Symbolic Execution](https://mlsanity.com/paper/2306.06755/) (2023)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [Rectifier: Code Translation with Corrector via LLMs](https://mlsanity.com/paper/2407.07472/) (2024)
- [Trustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmaps](https://mlsanity.com/paper/2309.04142/) (2023)
- [NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Prompts](https://mlsanity.com/paper/2405.04520/) (2024)
- [CodeRosetta: Pushing the Boundaries of Unsupervised Code Translation for Parallel Programming](https://mlsanity.com/paper/2410.20527/) (2024)
- [LangProp: A code optimization framework using Large Language Models applied to driving](https://mlsanity.com/paper/2401.10314/) (2024)
- [Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges](https://mlsanity.com/paper/2405.15604/) (2024)
- [Exploring the Robustness of Large Language Models for Solving Programming Problems](https://mlsanity.com/paper/2306.14583/) (2023)
- [InterTrans: Leveraging Transitive Intermediate Translations to Enhance LLM-based Code Translation](https://mlsanity.com/paper/2411.01063/) (2024)
- [CSR-Bench: Benchmarking LLM Agents in Deployment of Computer Science Research Repositories](https://mlsanity.com/paper/2502.06111/) (2025)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [Explain-then-Translate: An Analysis on Improving Program Translation with Self-generated Explanations](https://mlsanity.com/paper/2311.07070/) (2023)

## Abstract (arXiv)

With little to no parallel data available for programming languages, unsupervised methods are well-suited to source code translation. However, the majority of unsupervised machine translation approaches rely on back-translation, a method developed in the context of natural language translation and one that inherently involves training on noisy inputs. Unfortunately, source code is highly sensitive to small changes; a single token can result in compilation failures or erroneous programs, unlike natural languages where small inaccuracies may not change the meaning of a sentence. To address this issue, we propose to leverage an automated unit-testing system to filter out invalid translations, thereby creating a fully tested parallel corpus. We found that fine-tuning an unsupervised model with this filtered data set significantly reduces the noise in the translations so-generated, comfortably outperforming the state-of-the-art for all language pairs studied. In particular, for Java → Python and Python → C++ we outperform the best previous methods by more than 16% and 24% respectively, reducing the error rate by more than 35%.