# Conan-embedding: General Text Embedding with More and Better Negative Samples

arXiv: https://arxiv.org/abs/2408.15710 · HTML: https://mlsanity.com/paper/2408.15710/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 17 references that are in the map.
- First posted on arXiv 28 Aug 2024.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [InternLM2 Technical Report](https://mlsanity.com/paper/2403.17297/) (2024)
- [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://mlsanity.com/paper/2212.09741/) (2022)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [Matryoshka Representation Learning](https://mlsanity.com/paper/2205.13147/) (2022)
- [NV-Retriever: Improving text embedding models with effective hard-negative mining](https://mlsanity.com/paper/2407.15831/) (2024)
- [Challenging Decoder helps in Masked Auto-Encoder Pre-training for Dense Passage Retrieval](https://mlsanity.com/paper/2305.13197/) (2023)

## Built on by (most cited first, 17 of 17)

- [LLMs are Also Effective Embedding Models: An In-depth Overview](https://mlsanity.com/paper/2412.12591/) (2024)
- [Enhancing Few-Shot Vision-Language Classification with Large Multimodal Model Features](https://mlsanity.com/paper/2412.00142/) (2024)
- [SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Model](https://mlsanity.com/paper/2510.12709/) (2025)
- [IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation](https://mlsanity.com/paper/2603.04738/) (2026)
- [Training LLMs to be Better Text Embedders through Bidirectional Reconstruction](https://mlsanity.com/paper/2509.03020/) (2025)
- [Conan-Embedding-v2: Training an LLM from Scratch for Text Embeddings](https://mlsanity.com/paper/2509.12892/) (2025)
- [ListConRanker: A Contrastive Text Reranker with Listwise Encoding](https://mlsanity.com/paper/2501.07111/) (2025)
- [SEAL: Speech Embedding Alignment Learning for Speech Large Language Model with Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.02603/) (2025)
- [QZhou-Embedding Technical Report](https://mlsanity.com/paper/2508.21632/) (2025)
- [Dense Retrievers Can Fail on Simple Queries: Revealing The Granularity Dilemma of Embeddings](https://mlsanity.com/paper/2506.08592/) (2025)
- [Detecting LLM-Generated Spam Reviews by Integrating Language Model Embeddings and Graph Neural Network](https://mlsanity.com/paper/2510.01801/) (2025)
- [X-Intelligence 3.0: Training and Evaluating Reasoning LLM for Semiconductor Display](https://mlsanity.com/paper/2507.14430/) (2025)
- [Cropping outperforms dropout as an augmentation strategy for self-supervised training of text embeddings](https://mlsanity.com/paper/2508.03453/) (2025)
- [Checking Fact with Better Retrieval: Dynamic Contrastive Learning for Evidence Retrieval](https://mlsanity.com/paper/2605.27449/) (2026)
- [Ovis-Embedding: Pushing the Frontiers of Universal Omni-Modal Embeddings](https://mlsanity.com/paper/2609.25165/) (2026)
- [SRJudge: Empowering Large Language Models with Selective Reasoning for Fine-Grained Knowledge Concept Tagging](https://mlsanity.com/paper/2609.36982/) (2026)
- [LongHisDoc: A Comprehensive Benchmark for Chinese Long Historical Document Understanding](https://mlsanity.com/paper/2610.14531/) (2026)

## Abstract (arXiv)

With the growing popularity of RAG, the capabilities of embedding models are gaining increasing attention. Embedding models are primarily trained through contrastive loss learning, with negative examples being a key component. Previous work has proposed various hard negative mining strategies, but these strategies are typically employed as preprocessing steps. In this paper, we propose the conan-embedding model, which maximizes the utilization of more and higher-quality negative examples. Specifically, since the model's ability to handle preprocessed negative examples evolves during training, we propose dynamic hard negative mining method to expose the model to more challenging negative examples throughout the training process. Secondly, contrastive learning requires as many negative examples as possible but is limited by GPU memory constraints. Therefore, we use a Cross-GPU balancing Loss to provide more negative examples for embedding training and balance the batch size across multiple tasks. Moreover, we also discovered that the prompt-response pairs from LLMs can be used for embedding training. Our approach effectively enhances the capabilities of embedding models, currently ranking first on the Chinese leaderboard of Massive text embedding benchmark