mlsanityData to 7 Oct 2026

Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models

arXiv 2405.05374 ↗First posted 8 May 2024cs.CL cs.AI cs.IRcited by 60

Not on the map. Why?

Abstract (arXiv)

This report describes the training dataset creation and recipe behind the family of textttarctic-embed text embedding models (a set of five models ranging from 22 to 334 million parameters with weights open-sourced under an Apache-2 license). At the time of their release, each model achieved state-of-the-art retrieval accuracy for models of their size on the MTEB Retrieval leaderboard, with the largest model, arctic-embed-l outperforming closed source embedding models such as Cohere's embed-v3 and Open AI's text-embed-3-large. In addition to the details of our training recipe, we have provided several informative ablation studies, which we believe are the cause of our model performance.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 8 May 2024.

Builds on

20 of 31 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  7. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  8. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  9. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  10. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  11. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  12. 2023In-Context Retrieval-Augmented Language Modelscited by 694
8 more
  1. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  2. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
  3. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
  4. 2020MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformerscited by 240
  5. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  6. 2024Generative Representational Instruction Tuningcited by 227
  7. 2024Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERTcited by 21
  8. 2023DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Trainingcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 19; 2025: 27; 2026: 14.

60 papers.

Sort:

By area:

  1. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  2. 2024Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Datasetcited by 153
  3. 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 83
  4. 2024Arctic-Embed 2.0: Multilingual Retrieval Without Compromisecited by 57
  5. 2024Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Modelscited by 39
  6. 2024CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Rerankingcited by 37
  7. 2026SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agentscited by 37
  8. 2025Safety Pretraining: Toward the Next Generation of Safe AIcited by 35
  9. 2024Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Trackcited by 31
  10. 2024Initial Nugget Evaluation Results for the TREC 2024 RAG Track with the AutoNuggetizer Frameworkcited by 28
  11. 2024The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model designcited by 20
  12. 2024GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Searchcited by 20
38 more
  1. 2025SweRank: Software Issue Localization with Code Rankingcited by 20
  2. 2024The 2024 Foundation Model Transparency Indexcited by 18
  3. 2024Enhancing Q&A Text Retrieval with Ranking Models: Benchmarking, fine-tuning and deploying Rerankers for RAGcited by 16
  4. 2024JaColBERTv2.5: Optimising Multi-Vector Retrievers to Create State-of-the-Art Japanese Retrievers with Constrained Resourcescited by 15
  5. 2025Beyond Factual Accuracy: Evaluating Coverage of Diverse Factual Information in Long-form Text Generationcited by 15
  6. 2025FlexOlmo: Open Language Models for Flexible Data Usecited by 14
  7. 2024Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaborationcited by 10
  8. 2025MedGen: Unlocking Medical Video Generation by Scaling Granularly-annotated Medical Videoscited by 9
  9. 2024Model Editing for LLMs4Code: How Far are We?cited by 8
  10. 2024EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inferencecited by 6
  11. 2025Exploiting Leaderboards for Large-Scale Distribution of Malicious Modelscited by 5
  12. 2025DeRAG: Black-box Adversarial Attacks on Multiple Retrieval-Augmented Generation Applications via Prompt Injectioncited by 5
  13. 2025MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutchcited by 5
  14. 2025LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representationscited by 4
  15. 2025TalkCuts: A Large-Scale Dataset for Multi-Shot Human Speech Video Generationcited by 4
  16. 2025From Topology to Retrieval: Decoding Embedding Spaces with Unified Signaturescited by 4
  17. 2025Joint Selection for Large-Scale Pre-Training Data via Policy Gradient-based Mask Learningcited by 4
  18. 2024IRSC: A Zero-shot Evaluation Benchmark for Information Retrieval through Semantic Comprehension in Retrieval-Augmented Generation Scenarioscited by 3
  19. 2025Generative AI Enhanced Financial Risk Management Information Retrievalcited by 3
  20. 2025S-DAT: A Multilingual, GenAI-Driven Framework for Automated Divergent Thinking Assessmentcited by 3
  21. 2025On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Surveycited by 3
  22. 2025Removing Noise, not Finding Gold: Quality Filtering for Large-Scale Pretrainingcited by 3
  23. 2026DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Searchcited by 3
  24. 2024Qtok: A Comprehensive Framework for Evaluating Multilingual Tokenizer Quality in Large Language Modelscited by 2
  25. 2025Agentic Verification for Ambiguous Query Disambiguationcited by 2
  26. 2025Arctic Inference with Shift Parallelism: Fast and Efficient Open Source Inference System for Enterprise AIcited by 2
  27. 2025Retrieval Capabilities of Large Language Models Scale with Pretraining FLOPscited by 2
  28. 2025Chronological Passage Assembling in RAG framework for Temporal Question Answeringcited by 2
  29. 2025DMRetriever: A Family of Models for Improved Text Retrieval in Disaster Managementcited by 2
  30. 2026IRPAPERS: A Visual Document Benchmark for Scientific Retrieval and Question Answeringcited by 2
  31. 2024REFINE on Scarce Data: Retrieval Enhancement through Fine-Tuning via Model Fusion of Embedding Modelscited by 1
  32. 2025How to Evaluate Medical AIcited by 1
  33. 2025GigaEmbeddings: Efficient Russian Language Embedding Modelcited by 1
  34. 2026LMK > CLS: Landmark Pooling for Dense Embeddingscited by 1
  35. 2026Completing Missing Annotation: Multi-Agent Debate for Accurate and Scalable Relevant Assessment for IR Benchmarkscited by 1
  36. 2026Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMscited by 1
  37. 2026Towards Query-Agnostic RAG Evaluation via Query Coverage and Claim Verifiabilitycited by 1
  38. 2025Compressed Concatenation of Small Embedding Modelscited by 0

50 most cited of 60 All 60 (JSON)