# When Deep Learning Met Code Search

arXiv: https://arxiv.org/abs/1905.03813 · HTML: https://mlsanity.com/paper/1905.03813/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2019.

## Built on by (most cited first, 30 of 46)

- [ReACC: A Retrieval-Augmented Code Completion Framework](https://mlsanity.com/paper/2203.07722/) (2022)
- [CoSQA: 20,000+ Web Queries for Code Search and Question Answering](https://mlsanity.com/paper/2105.13239/) (2021)
- [DOBF: A Deobfuscation Pre-Training Objective for Programming Languages](https://mlsanity.com/paper/2102.07492/) (2021)
- [No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence](https://mlsanity.com/paper/2207.11680/) (2022)
- [Fault-Aware Neural Code Rankers](https://mlsanity.com/paper/2206.03865/) (2022)
- [On the Importance of Building High-quality Training Datasets for Neural Code Search](https://mlsanity.com/paper/2202.06649/) (2022)
- [CoDesc: A Large Code-Description Parallel Dataset](https://mlsanity.com/paper/2105.14220/) (2021)
- [CoCoSoDa: Effective Contrastive Learning for Code Search](https://mlsanity.com/paper/2204.03293/) (2022)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [Rewriting the Code: A Simple Method for Large Language Model Augmented Code Search](https://mlsanity.com/paper/2401.04514/) (2024)
- [AST-Probe: Recovering abstract syntax trees from hidden representations of pre-trained language models](https://mlsanity.com/paper/2206.11719/) (2022)
- [GrACE: Generation using Associated Code Edits](https://mlsanity.com/paper/2305.14129/) (2023)
- [GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search](https://mlsanity.com/paper/2111.02671/) (2021)
- [Code Search based on Context-aware Code Translation](https://mlsanity.com/paper/2202.08029/) (2022)
- [Rethinking Negative Pairs in Code Search](https://mlsanity.com/paper/2310.08069/) (2023)
- [Multimodal Representation for Neural Code Search](https://mlsanity.com/paper/2107.00992/) (2021)
- [CodeQueries: A Dataset of Semantic Queries over Code](https://mlsanity.com/paper/2209.08372/) (2022)
- [Poison Attack and Defense on Deep Source Code Processing Models](https://mlsanity.com/paper/2210.17029/) (2022)
- [Accelerating Code Search with Deep Hashing and Code Classification](https://mlsanity.com/paper/2203.15287/) (2022)
- [On the Effectiveness of Pretrained Models for API Learning](https://mlsanity.com/paper/2204.03498/) (2022)
- [Exploring Representation-Level Augmentation for Code Search](https://mlsanity.com/paper/2210.12285/) (2022)
- [Syntax and Domain Aware Model for Unsupervised Program Translation](https://mlsanity.com/paper/2302.03908/) (2023)
- [GenCodeSearchNet: A Benchmark Test Suite for Evaluating Generalization in Programming Language Understanding](https://mlsanity.com/paper/2311.09707/) (2023)
- [SPENCER: Self-Adaptive Model Distillation for Efficient Code Retrieval](https://mlsanity.com/paper/2508.00546/) (2025)
- [Cascaded Fast and Slow Models for Efficient Semantic Code Search](https://mlsanity.com/paper/2110.07811/) (2021)
- [A Survey of Deep Learning Models for Structural Code Understanding](https://mlsanity.com/paper/2205.01293/) (2022)
- [CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation](https://mlsanity.com/paper/2405.02355/) (2024)
- [deGraphCS: Embedding Variable-based Flow Graph for Neural Code Search](https://mlsanity.com/paper/2103.13020/) (2021)
- [AugmentedCode: Examining the Effects of Natural Language Resources in Code Retrieval Models](https://mlsanity.com/paper/2110.08512/) (2021)
- [Senatus -- A Fast and Accurate Code-to-Code Recommendation Engine](https://mlsanity.com/paper/2111.04473/) (2021)

## Abstract (arXiv)

There have been multiple recent proposals on using deep neural networks for code search using natural language. Common across these proposals is the idea of mathitembedding code and natural language queries, into real vectors and then using vector distance to approximate semantic correlation between code and the query. Multiple approaches exist for learning these embeddings, including mathitunsupervised techniques, which rely only on a corpus of code examples, and mathitsupervised techniques, which use an mathitaligned corpus of paired code and natural language descriptions. The goal of this supervision is to produce embeddings that are more similar for a query and the corresponding desired code snippet. Clearly, there are choices in whether to use supervised techniques at all, and if one does, what sort of network and training to use for supervision. This paper is the first to evaluate these choices systematically. To this end, we assembled implementations of state-of-the-art techniques to run on a common platform, training and evaluation corpora. To explore the design space in network complexity, we also introduced a new design point that is a mathitminimal supervision extension to an existing unsupervised technique. Our evaluation shows that: 1. adding supervision to an existing unsupervised technique can improve performance, though not necessarily by much; 2. simple networks for supervision can be more effective that more sophisticated sequence-based networks for code search; 3. while it is common to use docstrings to carry out supervision, there is a sizeable gap between the effectiveness of docstrings and a more query-appropriate supervision corpus. The evaluation dataset is now available at arXiv:1908.09804