# Commit2Vec: Learning Distributed Representations of Code Changes

arXiv: https://arxiv.org/abs/1911.07605 · HTML: https://mlsanity.com/paper/1911.07605/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2019.

## Built on by (most cited first, 12 of 12)

- [LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures](https://mlsanity.com/paper/2509.14252/) (2025)
- [Unsupervised Learning of General-Purpose Embeddings for Code Changes](https://mlsanity.com/paper/2106.02087/) (2021)
- [Dev2vec: Representing Domain Expertise of Developers in an Embedding Space](https://mlsanity.com/paper/2207.05132/) (2022)
- [PatchFinder: A Two-Phase Approach to Security Patch Tracing for Disclosed Vulnerabilities in Open-Source Software](https://mlsanity.com/paper/2407.17065/) (2024)
- [Deep Learning-Based Identification of Inconsistent Method Names: How Far Are We?](https://mlsanity.com/paper/2501.12617/) (2025)
- [Automated Mapping of Vulnerability Advisories onto their Fix Commits in Open Source Repositories](https://mlsanity.com/paper/2103.13375/) (2021)
- [Assessing the Effectiveness of Syntactic Structure to Learn Code Edit Representations](https://mlsanity.com/paper/2106.06110/) (2021)
- [Just-In-Time Software Defect Prediction via Bi-modal Change Representation Learning](https://mlsanity.com/paper/2410.12107/) (2024)
- [Securing the AI Supply Chain: What Can We Learn From Developer-Reported Security Issues and Solutions of AI Projects?](https://mlsanity.com/paper/2512.23385/) (2025)
- [Favia: Forensic Agent for Vulnerability-fix Identification and Analysis](https://mlsanity.com/paper/2602.12500/) (2026)
- [Detecting Security Fixes in Open-Source Repositories using Static Code Analyzers](https://mlsanity.com/paper/2105.03346/) (2021)
- [MQDD: Pre-training of Multimodal Question Duplicity Detection for Software Engineering Domain](https://mlsanity.com/paper/2203.14093/) (2022)

## Abstract (arXiv)

Deep learning methods, which have found successful applications in fields like image classification and natural language processing, have recently been applied to source code analysis too, due to the enormous amount of freely available source code (e.g., from open-source software repositories). In this work, we elaborate upon a state-of-the-art approach to the representation of source code that uses information about its syntactic structure, and we adapt it to represent source changes (i.e., commits). We use this representation to classify security-relevant commits. Because our method uses transfer learning (that is, we train a network on a "pretext task" for which abundant labeled data is available, and then we use such network for the target task of commit classification, for which fewer labeled instances are available), we studied the impact of pre-training the network using two different pretext tasks versus a randomly initialized model. Our results indicate that representations that leverage the structural information obtained through code syntax outperform token-based representations. Furthermore, the performance metrics obtained when pre-training on a loosely related pretext task with a very large dataset (>106 samples) were surpassed when pretraining on a smaller dataset (>104 samples) but for a pretext task that is more closely related to the target task.