# Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models

arXiv: https://arxiv.org/abs/2405.05374 · HTML: https://mlsanity.com/paper/2405.05374/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 8 May 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2010.08191/) (2020)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://mlsanity.com/paper/2402.01613/) (2024)
- [MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers](https://mlsanity.com/paper/2012.15828/) (2020)
- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://mlsanity.com/paper/2209.11755/) (2022)
- [Generative Representational Instruction Tuning](https://mlsanity.com/paper/2402.09906/) (2024)
- [Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT](https://mlsanity.com/paper/2402.07440/) (2024)
- [DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training](https://mlsanity.com/paper/2304.08480/) (2023)

## Built on by (most cited first, 30 of 61)

- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset](https://mlsanity.com/paper/2412.02595/) (2024)
- [NV-Retriever: Improving text embedding models with effective hard-negative mining](https://mlsanity.com/paper/2407.15831/) (2024)
- [Arctic-Embed 2.0: Multilingual Retrieval Without Compromise](https://mlsanity.com/paper/2412.04506/) (2024)
- [Safety Pretraining: Toward the Next Generation of Safe AI](https://mlsanity.com/paper/2504.16980/) (2025)
- [Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models](https://mlsanity.com/paper/2409.11136/) (2024)
- [SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agents](https://mlsanity.com/paper/2605.05726/) (2026)
- [CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking](https://mlsanity.com/paper/2412.01007/) (2024)
- [Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track](https://mlsanity.com/paper/2406.16828/) (2024)
- [Initial Nugget Evaluation Results for the TREC 2024 RAG Track with the AutoNuggetizer Framework](https://mlsanity.com/paper/2411.09607/) (2024)
- [The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design](https://mlsanity.com/paper/2408.12503/) (2024)
- [GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Search](https://mlsanity.com/paper/2412.20953/) (2024)
- [SweRank: Software Issue Localization with Code Ranking](https://mlsanity.com/paper/2505.07849/) (2025)
- [The 2024 Foundation Model Transparency Index](https://mlsanity.com/paper/2407.12929/) (2024)
- [Enhancing Q&A Text Retrieval with Ranking Models: Benchmarking, fine-tuning and deploying Rerankers for RAG](https://mlsanity.com/paper/2409.07691/) (2024)
- [JaColBERTv2.5: Optimising Multi-Vector Retrievers to Create State-of-the-Art Japanese Retrievers with Constrained Resources](https://mlsanity.com/paper/2407.20750/) (2024)
- [Beyond Factual Accuracy: Evaluating Coverage of Diverse Factual Information in Long-form Text Generation](https://mlsanity.com/paper/2501.03545/) (2025)
- [FlexOlmo: Open Language Models for Flexible Data Use](https://mlsanity.com/paper/2507.07024/) (2025)
- [Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaboration](https://mlsanity.com/paper/2410.08102/) (2024)
- [Model Editing for LLMs4Code: How Far are We?](https://mlsanity.com/paper/2411.06638/) (2024)
- [MedGen: Unlocking Medical Video Generation by Scaling Granularly-annotated Medical Videos](https://mlsanity.com/paper/2507.05675/) (2025)
- [EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference](https://mlsanity.com/paper/2410.12247/) (2024)
- [TalkCuts: A Large-Scale Dataset for Multi-Shot Human Speech Video Generation](https://mlsanity.com/paper/2510.07249/) (2025)
- [DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search](https://mlsanity.com/paper/2607.27178/) (2026)
- [Exploiting Leaderboards for Large-Scale Distribution of Malicious Models](https://mlsanity.com/paper/2507.08983/) (2025)
- [DeRAG: Black-box Adversarial Attacks on Multiple Retrieval-Augmented Generation Applications via Prompt Injection](https://mlsanity.com/paper/2507.15042/) (2025)
- [MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch](https://mlsanity.com/paper/2509.12340/) (2025)
- [From Topology to Retrieval: Decoding Embedding Spaces with Unified Signatures](https://mlsanity.com/paper/2511.22150/) (2025)
- [Joint Selection for Large-Scale Pre-Training Data via Policy Gradient-based Mask Learning](https://mlsanity.com/paper/2512.24265/) (2025)
- [LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representations](https://mlsanity.com/paper/2509.12539/) (2025)

## Abstract (arXiv)

This report describes the training dataset creation and recipe behind the family of textttarctic-embed text embedding models (a set of five models ranging from 22 to 334 million parameters with weights open-sourced under an Apache-2 license). At the time of their release, each model achieved state-of-the-art retrieval accuracy for models of their size on the MTEB Retrieval leaderboard, with the largest model, arctic-embed-l outperforming closed source embedding models such as Cohere's embed-v3 and Open AI's text-embed-3-large. In addition to the details of our training recipe, we have provided several informative ablation studies, which we believe are the cause of our model performance.