# Multimodal Representation for Neural Code Search

arXiv: https://arxiv.org/abs/2107.00992 · HTML: https://mlsanity.com/paper/2107.00992/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 66 references that are in the map.
- First posted on arXiv 2 Jul 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018)
- [A Convolutional Neural Network for Modelling Sentences](https://mlsanity.com/paper/1404.2188/) (2014)
- [Learning to Compare Image Patches via Convolutional Neural Networks](https://mlsanity.com/paper/1504.03641/) (2015)
- [Tree-to-tree Neural Networks for Program Translation](https://mlsanity.com/paper/1802.03691/) (2018)
- [Code Prediction by Feeding Trees to Transformers](https://mlsanity.com/paper/2003.13848/) (2020)
- [When Deep Learning Met Code Search](https://mlsanity.com/paper/1905.03813/) (2019)
- [Multi-Modal Attention Network Learning for Semantic Source Code Retrieval](https://mlsanity.com/paper/1909.13516/) (2019)
- [SourcererCC: Scaling Code Clone Detection to Big Code](https://mlsanity.com/paper/1512.06448/) (2015)
- [A General Path-Based Representation for Predicting Program Properties](https://mlsanity.com/paper/1803.09544/) (2018)
- [Aroma: Code Recommendation via Structural Code Search](https://mlsanity.com/paper/1812.01158/) (2018)
- [Opportunities and Challenges in Code Search Tools](https://mlsanity.com/paper/2011.02297/) (2020)
- [Identifying Corresponding Patches in SAR and Optical Images with a Pseudo-Siamese CNN](https://mlsanity.com/paper/1801.08467/) (2018)
- [Evaluating How Developers Use General-Purpose Web-Search for Code Retrieval](https://mlsanity.com/paper/1803.08612/) (2018)
- [Effective Reformulation of Query for Code Search using Crowdsourced Knowledge and Extra-Large Data Analytics](https://mlsanity.com/paper/1807.08798/) (2018)
- [Active Inductive Logic Programming for Code Search](https://mlsanity.com/paper/1812.05265/) (2018)

## Built on by (most cited first, 11 of 11)

- [CoCoSoDa: Effective Contrastive Learning for Code Search](https://mlsanity.com/paper/2204.03293/) (2022)
- [Assemble Foundation Models for Automatic Code Summarization](https://mlsanity.com/paper/2201.05222/) (2022)
- [Rewriting the Code: A Simple Method for Large Language Model Augmented Code Search](https://mlsanity.com/paper/2401.04514/) (2024)
- [Rethinking Negative Pairs in Code Search](https://mlsanity.com/paper/2310.08069/) (2023)
- [On the Effectiveness of Pretrained Models for API Learning](https://mlsanity.com/paper/2204.03498/) (2022)
- [Top Pass: Improve Code Generation by Pass@k-Maximized Code Ranking](https://mlsanity.com/paper/2408.05715/) (2024)
- [Exploring Representation-Level Augmentation for Code Search](https://mlsanity.com/paper/2210.12285/) (2022)
- [Abstract Syntax Tree for Programming Language Understanding and Representation: How Far Are We?](https://mlsanity.com/paper/2312.00413/) (2023)
- [CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation](https://mlsanity.com/paper/2405.02355/) (2024)
- [Refining Joint Text and Source Code Embeddings for Retrieval Task with Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2405.04126/) (2024)
- [CONCORD: Towards a DSL for Configurable Graph Code Representation](https://mlsanity.com/paper/2401.17967/) (2024)

## Abstract (arXiv)

Semantic code search is about finding semantically relevant code snippets for a given natural language query. In the state-of-the-art approaches, the semantic similarity between code and query is quantified as the distance of their representation in the shared vector space. In this paper, to improve the vector space, we introduce tree-serialization methods on a simplified form of AST and build the multimodal representation for the code data. We conduct extensive experiments using a single corpus that is large-scale and multi-language: CodeSearchNet. Our results show that both our tree-serialized representations and multimodal learning model improve the performance of code search. Last, we define intuitive quantification metrics oriented to the completeness of semantic and syntactic information of the code data, to help understand the experimental findings.