mlsanityData to 7 Oct 2026

CodeRetriever: Unimodal and Bimodal Contrastive Learning for Code Search

arXiv 2201.10866 ↗First posted 26 Jan 2022cs.CL cs.SEcited by 21

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose the CodeRetriever model, which learns the function-level code semantic representations through large-scale code-text contrastive pre-training. We adopt two contrastive learning schemes in CodeRetriever: unimodal contrastive learning and bimodal contrastive learning. For unimodal contrastive learning, we design an unsupervised learning approach to build semantic-related code pairs based on the documentation and function name. For bimodal contrastive learning, we leverage the documentation and in-line comments of code to build code-text pairs. Both contrastive objectives can fully leverage large-scale code corpus for pre-training. Extensive experimental results show that CodeRetriever achieves new state-of-the-art with significant improvement over existing code pre-trained models, on eleven domain/language-specific code search tasks with six programming languages in different code granularity (function-level, snippet-level and statement-level). These results demonstrate the effectiveness and robustness of CodeRetriever.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 32 of its 44 references that are in the map. First posted on arXiv 26 Jan 2022.

Builds on

32 of 44 references.

References · 32

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2017Billion-scale similarity search with GPUscited by 2,477
  6. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  7. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  8. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  9. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  10. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  11. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  12. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
20 more
  1. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  2. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  3. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  4. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  5. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  6. 2018Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflowcited by 165
  7. 2021Retrieval Augmented Code Generation and Summarizationcited by 148
  8. 2021Scaling Deep Contrastive Learning Batch Size under Memory Limited Setupcited by 123
  9. 2021Adversarial Retriever-Ranker for dense text retrievalcited by 82
  10. 2020Contrastive Code Representation Learningcited by 76
  11. 2021CoSQA: 20,000+ Web Queries for Code Search and Question Answeringcited by 74
  12. 2021CoTexT: Multi-task Learning with Code-Text Transformercited by 61
  13. 2020Self-Supervised Contrastive Learning for Code Retrieval and Summarization via Semantic-Preserving Transformationscited by 49
  14. 2019Learning and Evaluating Contextual Embedding of Source Codecited by 33
  15. 2020Exploring Software Naturalness through Neural Language Modelscited by 32
  16. 2018StaQC: A Systematically Mined Question-Code Dataset from Stack Overflowcited by 30
  17. 2017Query Expansion Based on Crowd Knowledge for Code Searchcited by 18
  18. 2020SCELMo: Source Code Embeddings from Language Modelscited by 16
  19. 2020Neural Code Search Revisited: Enhancing Code Snippet Retrieval through Natural Language Intentcited by 16
  20. 2019Neural Code Search Evaluation Datasetcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 10; 2026: 2.

21 papers.

Sort:

By area:

  1. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  2. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  3. 2024CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Rerankingcited by 37
  4. 2024Code Representation Learning At Scalecited by 26
  5. 2025Improving Code Localization with Repository Memorycited by 19
  6. 2025cAST: Enhancing Code Retrieval-Augmented Generation with Structural Chunking via Abstract Syntax Treecited by 17
  7. 2024ProCQA: A Large-scale Community-based Programming Question Answering Dataset for Code Searchcited by 12
  8. 2023GenCodeSearchNet: A Benchmark Test Suite for Evaluating Generalization in Programming Language Understandingcited by 6
  9. 2024Building A Coding Assistant via the Retrieval-Augmented Language Modelcited by 5
  10. 2024UniHGKR: Unified Instruction-aware Heterogeneous Knowledge Retrieverscited by 5
  11. 2025Functional Consistency of LLM Code Embeddings: A Self-Evolving Data Synthesis Framework for Benchmarkingcited by 4
  12. 2025On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Surveycited by 3
9 more
  1. 2025Multi-task retriever fine-tuning for domain-specific and efficient RAGcited by 2
  2. 2025OASIS: Order-Augmented Strategy for Improved Code Searchcited by 2
  3. 2025Zero-Shot Cross-Domain Code Search without Fine-Tuningcited by 2
  4. 2025SemanticForge: Repository-Level Code Generation through Semantic Knowledge Graphs and Constraint Satisfactioncited by 1
  5. 2024LSem2Vec: A Simple yet Effective Two-Stage Approach for Source Code Embeddingcited by 0
  6. 2025SaraCoder: Orchestrating Semantic and Structural Cues for Resource-Optimized Repository-Level Code Completioncited by 0
  7. 2025SEAL: Structure and Element Aware Learning to Improve Long Structured Document Retrievalcited by 0
  8. 2026XSearch: Explainable Code Search via Concept-to-Code Alignmentcited by 0
  9. 2026ExecRetrieval: Measuring the Functional-Correctness Gap in Code-Embedding Retrievalcited by 0