# OASIS: Order-Augmented Strategy for Improved Code Search

arXiv: https://arxiv.org/abs/2503.08161 · HTML: https://mlsanity.com/paper/2503.08161/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 11 Mar 2025.

## Builds on (most cited first, 28 of 28)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Retrieval Augmented Code Generation and Summarization](https://mlsanity.com/paper/2108.11601/) (2021)
- [DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings](https://mlsanity.com/paper/2204.10298/) (2022)
- [Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup](https://mlsanity.com/paper/2101.06983/) (2021)
- [ReACC: A Retrieval-Augmented Code Completion Framework](https://mlsanity.com/paper/2203.07722/) (2022)
- [An Unsupervised Sentence Embedding Method by Mutual Information Maximization](https://mlsanity.com/paper/2009.12061/) (2020)
- [Adversarial Retriever-Ranker for dense text retrieval](https://mlsanity.com/paper/2110.03611/) (2021)
- [Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer vision](https://mlsanity.com/paper/2109.08203/) (2021)
- [CoSQA: 20,000+ Web Queries for Code Search and Question Answering](https://mlsanity.com/paper/2105.13239/) (2021)
- [RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank](https://mlsanity.com/paper/2305.16726/) (2023)
- [CoCoSoDa: Effective Contrastive Learning for Code Search](https://mlsanity.com/paper/2204.03293/) (2022)
- [CodeRetriever: Unimodal and Bimodal Contrastive Learning for Code Search](https://mlsanity.com/paper/2201.10866/) (2022)
- [Query Expansion Based on Crowd Knowledge for Code Search](https://mlsanity.com/paper/1703.01443/) (2017)
- [Code Search: A Survey of Techniques for Finding Code](https://mlsanity.com/paper/2204.02765/) (2022)
- [How Far Can We Go with Practical Function-Level Program Repair?](https://mlsanity.com/paper/2404.12833/) (2024)
- [Ranking-Enhanced Unsupervised Sentence Representation Learning](https://mlsanity.com/paper/2209.04333/) (2022)
- [Rethinking Negative Pairs in Code Search](https://mlsanity.com/paper/2310.08069/) (2023)
- [Soft-Labeled Contrastive Pre-training for Function-level Code Representation](https://mlsanity.com/paper/2210.09597/) (2022)

## Built on by (most cited first, 2 of 2)

- [ReCode: Improving LLM-based Code Repair with Fine-Grained Retrieval-Augmented Generation](https://mlsanity.com/paper/2509.02330/) (2025)
- [Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence](https://mlsanity.com/paper/2505.11611/) (2025)

## Abstract (arXiv)

Code embeddings capture the semantic representations of code and are crucial for various code-related large language model (LLM) applications, such as code search. Previous training primarily relies on optimizing the InfoNCE loss by comparing positive natural language (NL)-code pairs with in-batch negatives. However, due to the sparse nature of code contexts, training solely by comparing the major differences between positive and negative pairs may fail to capture deeper semantic nuances. To address this issue, we propose a novel order-augmented strategy for improved code search (OASIS). It leverages order-based similarity labels to train models to capture subtle differences in similarity among negative pairs. Extensive benchmark evaluations demonstrate that our OASIS model significantly outperforms previous state-of-the-art models focusing solely on major positive-negative differences. It underscores the value of exploiting subtle differences among negative pairs with order labels for effective code embedding training.