# A parallel corpus of Python functions and documentation strings for automated code documentation and code generation

arXiv: https://arxiv.org/abs/1707.02275 · HTML: https://mlsanity.com/paper/1707.02275/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2017.

## Built on by (most cited first, 30 of 57)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution](https://mlsanity.com/paper/2401.03065/) (2024)
- [OctoPack: Instruction Tuning Code Large Language Models](https://mlsanity.com/paper/2308.07124/) (2023)
- [Natural Language to Code Translation with Execution](https://mlsanity.com/paper/2204.11454/) (2022)
- [Retrieval-Augmented Generation for Code Summarization via Hybrid GNN](https://mlsanity.com/paper/2006.05405/) (2020)
- [Leveraging Automated Unit Tests for Unsupervised Code Translation](https://mlsanity.com/paper/2110.06773/) (2021)
- [CoSQA: 20,000+ Web Queries for Code Search and Question Answering](https://mlsanity.com/paper/2105.13239/) (2021)
- [Out of the BLEU: how should we assess quality of the Code Generation models?](https://mlsanity.com/paper/2208.03133/) (2022)
- [xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval](https://mlsanity.com/paper/2303.03004/) (2023)
- [HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation](https://mlsanity.com/paper/2412.21199/) (2024)
- [Fault-Aware Neural Code Rankers](https://mlsanity.com/paper/2206.03865/) (2022)
- [Code Structure Guided Transformer for Source Code Summarization](https://mlsanity.com/paper/2104.09340/) (2021)
- [CodeQA: A Question Answering Dataset for Source Code Comprehension](https://mlsanity.com/paper/2109.08365/) (2021)
- [On the Importance of Building High-quality Training Datasets for Neural Code Search](https://mlsanity.com/paper/2202.06649/) (2022)
- [The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Python](https://mlsanity.com/paper/2305.15507/) (2023)
- [A Survey on Machine Learning Techniques for Source Code Analysis](https://mlsanity.com/paper/2110.09610/) (2021)
- [RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation](https://mlsanity.com/paper/2412.17744/) (2024)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Code](https://mlsanity.com/paper/2205.11739/) (2022)
- [The Vault: A Comprehensive Multilingual Dataset for Advancing Code Understanding and Generation](https://mlsanity.com/paper/2305.06156/) (2023)
- [Analyzing the Performance of Large Language Models on Code Summarization](https://mlsanity.com/paper/2404.08018/) (2024)
- [An Extractive-and-Abstractive Framework for Source Code Summarization](https://mlsanity.com/paper/2206.07245/) (2022)
- [In Line with Context: Repository-Level Code Generation via Context Inlining](https://mlsanity.com/paper/2601.00376/) (2026)
- [Python Code Generation by Asking Clarification Questions](https://mlsanity.com/paper/2212.09885/) (2022)
- [Source Code Summarization with Structural Relative Position Guided Transformer](https://mlsanity.com/paper/2202.06521/) (2022)
- [M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarization](https://mlsanity.com/paper/2203.09707/) (2022)
- [Aligning Offline Metrics and Human Judgments of Value for Code Generation Models](https://mlsanity.com/paper/2210.16494/) (2022)
- [Code Recommendation for Open Source Software Developers](https://mlsanity.com/paper/2210.08332/) (2022)
- [Free and Customizable Code Documentation with LLMs: A Fine-Tuning Approach](https://mlsanity.com/paper/2412.00726/) (2024)

## Abstract (arXiv)

Automated documentation of programming source code and automated code generation from natural language are challenging tasks of both practical and scientific interest. Progress in these areas has been limited by the low availability of parallel corpora of code and natural language descriptions, which tend to be small and constrained to specific domains. In this work we introduce a large and diverse parallel corpus of a hundred thousands Python functions with their documentation strings ("docstrings") generated by scraping open source repositories on GitHub. We describe baseline results for the code documentation and code generation tasks obtained by neural machine translation. We also experiment with data augmentation techniques to further increase the amount of training data. We release our datasets and processing scripts in order to stimulate research in these areas.