# CC2Vec: Distributed Representations of Code Changes

arXiv: https://arxiv.org/abs/2003.05620 · HTML: https://mlsanity.com/paper/2003.05620/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2020.

## Built on by (most cited first, 30 of 39)

- [CURE: Code-Aware Neural Machine Translation for Automatic Program Repair](https://mlsanity.com/paper/2103.00073/) (2021)
- [Code Generation Tools (Almost) for Free? A Study of Few-Shot, Pre-Trained Language Models on Code](https://mlsanity.com/paper/2206.01335/) (2022)
- [LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures](https://mlsanity.com/paper/2509.14252/) (2025)
- [A Survey on Machine Learning Techniques for Source Code Analysis](https://mlsanity.com/paper/2110.09610/) (2021)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [On the Evaluation of Commit Message Generation Models: An Experimental Study](https://mlsanity.com/paper/2107.05373/) (2021)
- [Trustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmaps](https://mlsanity.com/paper/2309.04142/) (2023)
- [Learning Structural Edits via Incremental Tree Transformations](https://mlsanity.com/paper/2101.12087/) (2021)
- [From Commit Message Generation to History-Aware Commit Message Completion](https://mlsanity.com/paper/2308.07655/) (2023)
- [DeepCVA: Automated Commit-level Vulnerability Assessment with Deep Multi-task Learning](https://mlsanity.com/paper/2108.08041/) (2021)
- [Invalidator: Automated Patch Correctness Assessment via Semantic and Syntactic Reasoning](https://mlsanity.com/paper/2301.01113/) (2023)
- [Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey](https://mlsanity.com/paper/2310.17903/) (2023)
- [Is this Change the Answer to that Problem? Correlating Descriptions of Bug and Code Changes for Evaluating Patch Correctness](https://mlsanity.com/paper/2208.04125/) (2022)
- [On the Usage of Continual Learning for Out-of-Distribution Generalization in Pre-trained Language Models of Code](https://mlsanity.com/paper/2305.04106/) (2023)
- [On the Use of Fine-grained Vulnerable Code Statements for Software Vulnerability Assessment Models](https://mlsanity.com/paper/2203.08417/) (2022)
- [Predicting Patch Correctness Based on the Similarity of Failing Test Cases](https://mlsanity.com/paper/2107.13296/) (2021)
- [Multi-Granularity Detector for Vulnerability Fixes](https://mlsanity.com/paper/2305.13884/) (2023)
- [Generating Informative CVE Description From ExploitDB Posts by Extractive Summarization](https://mlsanity.com/paper/2101.01431/) (2021)
- [Unsupervised Learning of General-Purpose Embeddings for Code Changes](https://mlsanity.com/paper/2106.02087/) (2021)
- [ApacheJIT: A Large Dataset for Just-In-Time Defect Prediction](https://mlsanity.com/paper/2203.00101/) (2022)
- [CCRep: Learning Code Change Representations via Pre-Trained Code Model and Query Back](https://mlsanity.com/paper/2302.03924/) (2023)
- [A study on the impact of pre-trained model on Just-In-Time defect prediction](https://mlsanity.com/paper/2309.02317/) (2023)
- [COMET: Generating Commit Messages using Delta Graph Context Representation](https://mlsanity.com/paper/2402.01841/) (2024)
- [Repo2Vec: A Comprehensive Embedding Approach for Determining Repository Similarity](https://mlsanity.com/paper/2107.05112/) (2021)
- [Bridging Expert Knowledge with Deep Learning Techniques for Just-In-Time Defect Prediction](https://mlsanity.com/paper/2403.11079/) (2024)
- [An Exploratory Literature Study on Sharing and Energy Use of Language Models for Source Code](https://mlsanity.com/paper/2307.02443/) (2023)
- [Just-In-Time Software Defect Prediction via Bi-modal Change Representation Learning](https://mlsanity.com/paper/2410.12107/) (2024)
- [Favia: Forensic Agent for Vulnerability-fix Identification and Analysis](https://mlsanity.com/paper/2602.12500/) (2026)
- [Rethinking Code Review in the Age of AI: A Vision for Agentic Code Review](https://mlsanity.com/paper/2605.17548/) (2026)
- [Predicting Line-Level Defects by Capturing Code Contexts with Hierarchical Transformers](https://mlsanity.com/paper/2312.11889/) (2023)

## Abstract (arXiv)

Existing work on software patches often use features specific to a single task. These works often rely on manually identified features, and human effort is required to identify these features for each task. In this work, we propose CC2Vec, a neural network model that learns a representation of code changes guided by their accompanying log messages, which represent the semantic intent of the code changes. CC2Vec models the hierarchical structure of a code change with the help of the attention mechanism and uses multiple comparison functions to identify the differences between the removed and added code. To evaluate if CC2Vec can produce a distributed representation of code changes that is general and useful for multiple tasks on software patches, we use the vectors produced by CC2Vec for three tasks: log message generation, bug fixing patch identification, and just-in-time defect prediction. In all tasks, the models using CC2Vec outperform the state-of-the-art techniques.