# Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systems

arXiv: https://arxiv.org/abs/2505.18366 · HTML: https://mlsanity.com/paper/2505.18366/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 63 references that are in the map.
- First posted on arXiv 23 May 2025.

## Builds on (most cited first, 30 of 30)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2010.08191/) (2020)
- [mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval](https://mlsanity.com/paper/2407.19669/) (2024)
- [Multi-Stage Document Ranking with BERT](https://mlsanity.com/paper/1910.14424/) (2019)
- [CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims](https://mlsanity.com/paper/2012.00614/) (2020)
- [Condenser: a Pre-training Architecture for Dense Retrieval](https://mlsanity.com/paper/2104.08253/) (2021)
- [Optimizing Dense Retrieval Model Training with Hard Negatives](https://mlsanity.com/paper/2104.08051/) (2021)
- [jina-embeddings-v3: Multilingual Embeddings With Task LoRA](https://mlsanity.com/paper/2409.10173/) (2024)
- [Re2G: Retrieve, Rerank, Generate](https://mlsanity.com/paper/2207.06300/) (2022)
- [NV-Retriever: Improving text embedding models with effective hard-negative mining](https://mlsanity.com/paper/2407.15831/) (2024)
- [CEDR: Contextualized Embeddings for Document Ranking](https://mlsanity.com/paper/1904.07094/) (2019)
- [Dense Hierarchical Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2110.15439/) (2021)
- [SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for Enterprise Use](https://mlsanity.com/paper/2505.17332/) (2025)
- [LLM for Barcodes: Generating Diverse Synthetic Data for Identity Documents](https://mlsanity.com/paper/2411.14962/) (2024)
- [Enhancing Document AI Data Generation Through Graph-Based Synthetic Layouts](https://mlsanity.com/paper/2412.03590/) (2024)
- [Tokenization Matters: Improving Zero-Shot NER for Indic Languages](https://mlsanity.com/paper/2504.16977/) (2025)
- [GripRank: Bridging the Gap between Retrieval and Generation via the Generative Knowledge Improved Passage Ranking](https://mlsanity.com/paper/2305.18144/) (2023)
- [TriSampler: A Better Negative Sampling Principle for Dense Retrieval](https://mlsanity.com/paper/2402.11855/) (2024)
- [FS-DAG: Few Shot Domain Adapting Graph Networks for Visually Rich Document Understanding](https://mlsanity.com/paper/2505.17330/) (2025)
- [Clinical QA 2.0: Multi-Task Learning for Answer Extraction and Categorization](https://mlsanity.com/paper/2502.13108/) (2025)
- [Generating Synthetic Documents for Cross-Encoder Re-Rankers: A Comparative Study of ChatGPT and Human Experts](https://mlsanity.com/paper/2305.02320/) (2023)
- [Enhancing Retrieval Performance: An Ensemble Approach For Hard Negative Mining](https://mlsanity.com/paper/2411.02404/) (2024)

## Built on by (most cited first, 14 of 14)

- [Aligning LLMs for Multilingual Consistency in Enterprise Applications](https://mlsanity.com/paper/2509.23659/) (2025)
- [Can LLMs Narrate Tabular Data? An Evaluation Framework for Natural Language Representations of Text-to-SQL System Outputs](https://mlsanity.com/paper/2510.23854/) (2025)
- [Who's Asking? Investigating Bias Through the Lens of Disability Framed Queries in LLMs](https://mlsanity.com/paper/2508.15831/) (2025)
- [IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languages](https://mlsanity.com/paper/2602.16832/) (2026)
- [JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agents](https://mlsanity.com/paper/2604.19821/) (2026)
- [FlexDoc: Parameterized Sampling for Diverse Multilingual Synthetic Documents for Training Document Understanding Models](https://mlsanity.com/paper/2510.02133/) (2025)
- [LLM-Guided Lifecycle-Aware Clustering of Multi-Turn Customer Support Conversations](https://mlsanity.com/paper/2601.04388/) (2026)
- [MT-OSC: Path for LLMs that Get Lost in Multi-Turn Conversation](https://mlsanity.com/paper/2604.08782/) (2026)
- [Do Image-Text Metrics Respect Semantic Invariances?](https://mlsanity.com/paper/2605.24702/) (2026)
- [PCRI: Measuring Context Robustness in Multimodal Models for Enterprise Applications](https://mlsanity.com/paper/2509.23879/) (2025)
- [Cross-Genre Authorship Attribution via LLM-Based Retrieve-and-Rerank](https://mlsanity.com/paper/2510.16819/) (2025)
- [Who Gets Left Behind? Auditing Disability Inclusivity in Large Language Models](https://mlsanity.com/paper/2509.00963/) (2025)
- [CroPS: Improving Dense Retrieval with Cross-Perspective Positive Samples in Short-Video Search](https://mlsanity.com/paper/2511.15443/) (2025)
- [From Codebase to Culprit (C2C): Reducing the Search Space for Bugs with Semantic Retrieval and Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2609.38402/) (2026)

## Abstract (arXiv)

Enterprise search systems often struggle to retrieve accurate, domain-specific information due to semantic mismatches and overlapping terminologies. These issues can degrade the performance of downstream applications such as knowledge management, customer support, and retrieval-augmented generation agents. To address this challenge, we propose a scalable hard-negative mining framework tailored specifically for domain-specific enterprise data. Our approach dynamically selects semantically challenging but contextually irrelevant documents to enhance deployed re-ranking models. Our method integrates diverse embedding models, performs dimensionality reduction, and uniquely selects hard negatives, ensuring computational efficiency and semantic precision. Evaluation on our proprietary enterprise corpus (cloud services domain) demonstrates substantial improvements of 15% in MRR@3 and 19% in MRR@10 compared to state-of-the-art baselines and other negative sampling techniques. Further validation on public domain-specific datasets (FiQA, Climate Fever, TechQA) confirms our method's generalizability and readiness for real-world applications.