What Tracks Hard-Negative Transfer? A Study of Lexical and Semantic Signals
Not on the map. Why?
Abstract (arXiv)
Which properties of hard-negative supervision track dense-retriever transfer before training? We compare lexical noncoverage, frozenencoder similarity statistics, hardness proxies, and combinations of these signals across BM25, dense, LLM-generated, and hybrid sources for MS MARCO training. Evaluation uses three retriever backbones and eleven BEIR datasets. Mean lexical residuality, an IDF-weighted measure of query-token noncoverage, reproduces the aggregate transfer ordering in all six backbone and setting blocks. The individual semantic statistics recover none of the complete orderings, and combining them with the lexical signal does not improve the observed rank agreement. After approximately matching BM25, dense, and LLM diagnostic samples for lexical residuality, none of the tested semantic statistics recovers the observed LLM > Dense > BM25 transfer ordering across three backbones, three matching resolutions, and 100 repetitions. Constructed controls further show that unrelated and false-negative-shaped passages can exhibit favorable lexical and semantic statistics. The findings identify a simple lexical association in this setting while distinguishing source level transfer agreement from the reliability of individual negatives.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 40 references that are in the map. First posted on arXiv 22 Mar 2026.
Builds on
26 of 40 references.
References · 26
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
- 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
- 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
- 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
- 2020Debiased Contrastive Learningcited by 356
14 more
- 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
- 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 235
- 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
- 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 155
- 2021Scaling Deep Contrastive Learning Batch Size under Memory Limited Setupcited by 127
- 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
- 2019Variational Bayesian Optimal Experimental Designcited by 116
- 2024BM25S: Orders of magnitude faster lexical search via eager sparse scoringcited by 88
- 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 84
- 2024SyNeg: LLM-Driven Synthetic Hard-Negatives for Dense Retrievalcited by 14
- 2024TriSampler: A Better Negative Sampling Principle for Dense Retrievalcited by 10
- 2023Mitigating the Impact of False Negatives in Dense Retrieval with Contrastive Confidence Regularizationcited by 4
- 2025Don't Retrieve, Generate: Prompting LLMs for Synthetic Training Data in Dense Retrievalcited by 3
- 2025BiCA: Effective Biomedical Dense Retrieval with Citation-Aware Hard Negativescited by 3
Built on by
Not cited yet.