DREditor: An Time-efficient Approach for Building a Domain-specific Dense Retrieval Model
Not on the map. Why?
Abstract (arXiv)
Deploying dense retrieval models efficiently is becoming increasingly important across various industries. This is especially true for enterprise search services, where customizing search engines to meet the time demands of different enterprises in different domains is crucial. Motivated by this, we develop a time-efficient approach called DREditor to edit the matching rule of an off-the-shelf dense retrieval model to suit a specific domain. This is achieved by directly calibrating the output embeddings of the model using an efficient and effective linear mapping. This mapping is powered by an edit operator that is obtained by solving a specially constructed least squares problem. Compared to implicit rule modification via long-time finetuning, our experimental results show that DREditor provides significant advantages on different domain-specific datasets, dataset sources, retrieval models, and computing devices. It consistently enhances time efficiency by 100-300 times while maintaining comparable or even superior retrieval performance. In a broader context, we take the first step to introduce a novel embedding calibration approach for the retrieval task, filling the technical blank in the current field of embedding calibration. This approach also paves the way for building domain-specific dense retrieval models efficiently and inexpensively.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 26 of its 55 references that are in the map. First posted on arXiv 23 Jan 2024.
Builds on
26 of 55 references.
References · 26
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2019Language Models as Knowledge Bases?cited by 1,922
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
- 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
- 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
- 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
- 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 351
- 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
14 more
- 2022On the Effectiveness of Parameter-Efficient Fine-Tuningcited by 140
- 2021On Calibration and Out-of-domain Generalizationcited by 134
- 2020Rewriting a Deep Generative Modelcited by 105
- 2020Language Models are Open Knowledge Graphscited by 95
- 2021Editing a classifier by rewriting its prediction rulescited by 64
- 2022Kformer: Knowledge Injection in Transformer Feed-Forward Layerscited by 37
- 2022A Thorough Examination on Zero-shot Dense Retrievalcited by 21
- 2020MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scalecited by 20
- 2022COCO-DR: Combating Distribution Shifts in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learningcited by 19
- 2021Predicting Efficiency/Effectiveness Trade-offs for Dense vs. Sparse Retrieval Strategy Selectioncited by 15
- 2019Domain Adaptation for Enterprise Email Searchcited by 7
- 2020Scheduling Beyond CPUs for HPCcited by 5
- 2019Emotional Embeddings: Refining Word Embeddings to Capture Emotional Content of Wordscited by 4
- 2017Retrofitting Distributional Embeddings to Knowledge Graphs with Functional Relationscited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.