mlsanityData to 14 Oct 2026

CCT-Code: Cross-Consistency Training for Multilingual Clone Detection and Code Search

arXiv 2305.11626 ↗First posted 19 May 2023cs.CL cs.SEcited by 4

Not on the map. Why?

Abstract (arXiv)

We consider the well-known and important tasks of clone detection and information retrieval for source code. The most standard setup is to search clones inside the same language code snippets. But it is also useful to find code snippets with identical behaviour in different programming languages. Nevertheless multi- and cross-lingual clone detection has been little studied in literature. We present a novel training procedure, cross-consistency training (CCT) leveraging cross-lingual similarity, that we apply to train language models on source code in various programming languages. We show that this training is effective both for encoder- and decoder-based models. The trained encoder-based CCT-LM model achieves a new state of the art on POJ-104 (monolingual C++ clone detection benchmark) with 96.73% MAP and AdvTest (monolingual Python code search benchmark) with 47.18% MRR. The decoder-based CCT-LM model shows comparable performance in these tasks. In addition, we formulate the multi- and cross-lingual clone detection problem and present XCD, a new benchmark dataset produced from CodeForces submissions.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 32 of its 57 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

32 of 57 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  6. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  7. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  8. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  9. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
  10. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  11. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  12. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
20 more
  1. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  2. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  3. 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 272
  4. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  5. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  6. 2020Detecting Code Clones with Graph Neural Networkand Flow-Augmented Abstract Syntax Treecited by 69
  7. 2019Cross-Lingual Natural Language Generation via Pre-Trainingcited by 63
  8. 2021SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representationcited by 63
  9. 2020Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answeringcited by 41
  10. 2015SourcererCC: Scaling Code Clone Detection to Big Codecited by 40
  11. 2019Cross-Lingual Training for Automatic Question Generationcited by 23
  12. 2021CoDesc: A Large Code-Description Parallel Datasetcited by 23
  13. 2018Unsupervised Cross-lingual Transfer of Word Embedding Spacescited by 16
  14. 2021X-METRA-ADA: Cross-lingual Meta-Transfer Learning Adaptation to Natural Language Understanding and Question Answeringcited by 10
  15. 2019Cross-Lingual Transfer Learning for Question Answeringcited by 6
  16. 2017NLP2Code: Code Snippet Content Assist via Natural Language Taskscited by 4
  17. 2021Bilingual alignment transfers to multilingual alignment for unsupervised parallel text miningcited by 4
  18. 2021Cascaded Fast and Slow Models for Efficient Semantic Code Searchcited by 4
  19. 2020Towards Semantic Clone Detection via Probabilistic Software Modelingcited by 1
  20. 2022Addressing Leakage in Self-Supervised Contextualized Code Retrievalcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 0; 2026: 2.

4 papers.

  1. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  2. 2026IDLM: Inverse-distilled Diffusion Language Modelscited by 8
  3. 2024The Struggles of LLMs in Cross-lingual Code Clone Detectioncited by 3
  4. 2026Standing on the Shoulders of Giants: Stabilized Knowledge Distillation for Cross--Language Code Clone Detectioncited by 0