Evaluation of Contrastive Learning with Various Code Representations for Code Clone Detection
Not on the map. Why?
Abstract (arXiv)
Code clones are pairs of code snippets that implement similar functionality. Clone detection is a fundamental branch of automatic source code comprehension, having many applications in refactoring recommendation, plagiarism detection, and code summarization. A particularly interesting case of clone detection is the detection of semantic clones, i.e., code snippets that have the same functionality but significantly differ in implementation. A promising approach to detecting semantic clones is contrastive learning (CL), a machine learning paradigm popular in computer vision but not yet commonly adopted for code processing. Our work aims to evaluate the most popular CL algorithms combined with three source code representations on two tasks. The first task is code clone detection, which we evaluate on the POJ-104 dataset containing implementations of 104 algorithms. The second task is plagiarism detection. To evaluate the models on this task, we introduce CodeTransformator, a tool for transforming source code. We use it to create a dataset that mimics plagiarised code based on competitive programming solutions. We trained nine models for both tasks and compared them with six existing approaches, including traditional tools and modern pre-trained neural models. The results of our evaluation show that proposed models perform diversely in each task, however the performance of the graph-based models is generally above the others. Among CL algorithms, SimCLR and SwAV lead to better results, while Moco is the most robust approach. Our code and trained models are available at https://doi.org/10.5281/zenodo.6360627, https://doi.org/10.5281/zenodo.5596345.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 41 of its 61 references that are in the map. First posted on arXiv 17 Jun 2022.
Builds on
41 of 61 references.
References · 41
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
- 2020Supervised Contrastive Learningcited by 2,829
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
- 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
29 more
- 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
- 2020A Primer in BERTology: What we know about how BERT workscited by 815
- 2020Contrastive Learning with Hard Negative Samplescited by 628
- 2020A Survey on Contrastive Self-supervised Learningcited by 554
- 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
- 2020A Metric Learning Reality Checkcited by 321
- 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 272
- 2020Unsupervised Translation of Programming Languagescited by 263
- 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 251
- 2020DeeperGCN: All You Need to Train Deeper GCNscited by 250
- 2019Graph Matching Networks for Learning the Similarity of Graph Structured Objectscited by 205
- 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
- 2018code2seq: Generating Sequences from Structured Representations of Codecited by 179
- 2019Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAScited by 83
- 2020Contrastive Code Representation Learningcited by 76
- 2020Detecting Code Clones with Graph Neural Networkand Flow-Augmented Abstract Syntax Treecited by 69
- 2021Language-Agnostic Representation Learning of Source Code from Structure and Contextcited by 63
- 2018Neural Code Comprehension: A Learnable Representation of Code Semanticscited by 59
- 2018Structured Neural Summarizationcited by 54
- 2021A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectivescited by 41
- 2019Misleading Authorship Attribution of Source Code using Adversarial Learningcited by 33
- 2020InferCode: Self-Supervised Learning of Code Representations by Predicting Subtreescited by 23
- 2020funcGNN: A Graph Neural Network Approach to Program Similaritycited by 16
- 2020The Effectiveness of Supervised Machine Learning Algorithms in Predicting Software Refactoringcited by 9
- 2019Import2vec - Learning Embeddings for Software Librariescited by 7
- 2020Mossad: Defeating Software Plagiarism Detectioncited by 5
- 2021UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learningcited by 4
- 2020A Survey on the Evaluation of Clone Detection Performance and Benchmarkingcited by 3
- 2021A Comparison of Code Embeddings and Beyondcited by 3
Built on by
Citing papers per year: 2023: 2; 2024: 0; 2025: 1.
3 papers.