mlsanityData to 7 Oct 2026

Nomic Embed: Training a Reproducible Long Context Text Embedder

arXiv 2402.01613 ↗First posted 2 Feb 2024cs.CL cs.AIcited by 255

Not on the map. Why?

Cited by 55 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This technical report describes the training of nomic-embed-text-v1, the first fully reproducible, open-source, open-weights, open-data, 8192 context length English text embedding model that outperforms both OpenAI Ada-002 and OpenAI text-embedding-3-small on the short-context MTEB benchmark and the long context LoCo benchmark. We release the training code and model weights under an Apache 2.0 license. In contrast with other open-source models, we release the full curated training data and code that allows for full replication of nomic-embed-text-v1. You can find code and data to replicate the model at https://github.com/nomic-ai/contrastors.

Key facts. Cited by 255 papers in the map (data through 7 Oct 2026). Builds on 51 of its 77 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

51 of 77 references.

References · 51

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  9. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  10. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  11. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
38 more
  1. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  2. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  3. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  6. 2020GLU Variants Improve Transformercited by 1,724
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  8. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  9. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  10. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  11. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  12. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  13. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  14. 2017Mixed Precision Trainingcited by 1,135
  15. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  16. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  17. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  18. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  19. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  20. 2019ELI5: Long Form Question Answeringcited by 567
  21. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  22. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  23. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  24. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  25. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  26. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  27. 2023Improving Text Embeddings with Large Language Modelscited by 295
  28. 2018WikiHow: A Large Scale Text Summarization Datasetcited by 203
  29. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  30. 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
  31. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  32. 2019BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarizationcited by 155
  33. 2021Scaling Deep Contrastive Learning Batch Size under Memory Limited Setupcited by 123
  34. 2023Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documentscited by 103
  35. 2022SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrievalcited by 90
  36. 2022Cramming: Training a Language Model on a Single GPU in One Daycited by 81
  37. 2021GooAQ: Open Question Answering with Diverse Answer Typescited by 53
  38. 2019AmazonQA: A Review-Based Question Answering Taskcited by 43

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 53; 2025: 108; 2026: 94.

255 papers.

Sort:

By area:

  1. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  2. 2024RLHF Workflow: From Reward Modeling to Online RLHFcited by 265
  3. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 233
  4. 2024AutoSurvey: Large Language Models Can Automatically Write Surveyscited by 116
  5. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
  6. 2025ReasonIR: Training Retrievers for Reasoning Taskscited by 76
  7. 2024Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Modelscited by 70
  8. 2024Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Modelscited by 60
  9. 2024Arctic-Embed 2.0: Multilingual Retrieval Without Compromisecited by 57
  10. 2025Knowledge Graph-Guided Retrieval Augmented Generationcited by 57
  11. 2024ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilitiescited by 52
  12. 2025RAG vs. GraphRAG: A Systematic Evaluation and Key Insightscited by 51
38 more
  1. 2024LongEmbed: Extending Embedding Models for Long Context Retrievalcited by 47
  2. 2024Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrievalcited by 45
  3. 2024Binary Classifier Optimization for Large Language Model Alignmentcited by 44
  4. 2024BioRAG: A RAG-LLM Framework for Biological Question Reasoningcited by 42
  5. 2025Griffin: Towards a Graph-Centric Relational Database Foundation Modelcited by 41
  6. 2025mmBERT: A Modern Multilingual Encoder with Annealed Language Learningcited by 39
  7. 2024CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Rerankingcited by 37
  8. 2025Seq vs Seq: An Open Suite of Paired Encoders and Decoderscited by 34
  9. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32
  10. 2025LettuceDetect: A Hallucination Detection Framework for RAG Applicationscited by 32
  11. 2024BERGEN: A Benchmarking Library for Retrieval-Augmented Generationcited by 31
  12. 2024Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreakingcited by 30
  13. 2025ArchRAG: Attributed Community-based Hierarchical Retrieval-Augmented Generationcited by 27
  14. 2025Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representationcited by 25
  15. 2024Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERTcited by 21
  16. 2024High-Dimension Human Value Representation in Large Language Modelscited by 20
  17. 2024Towards Data-Centric RLHF: Simple Metrics for Preference Dataset Comparisoncited by 20
  18. 2025SweRank: Software Issue Localization with Code Rankingcited by 20
  19. 2024Recent advances in text embedding: A Comprehensive Review of Top-Performing Methods on the MTEB Benchmarkcited by 19
  20. 2024Nomic Embed Vision: Expanding the Latent Spacecited by 19
  21. 2024Contextual Document Embeddingscited by 19
  22. 2024Better Instruction-Following Through Minimum Bayes Riskcited by 19
  23. 2025DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrieverscited by 19
  24. 2025ModernVBERT: Towards Smaller Visual Document Retrieverscited by 19
  25. 2024Embedding And Clustering Your Data Can Improve Contrastive Pretrainingcited by 17
  26. 2025PyLate: Flexible Training and Retrieval for Late Interaction Modelscited by 17
  27. 2024Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generationcited by 16
  28. 2025RAS: Retrieval-And-Structuring for Knowledge-Intensive LLM Generationcited by 16
  29. 2024Tracking the perspectives of interacting language modelscited by 14
  30. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  31. 2024APTNESS: Incorporating Appraisal Theory and Emotion Support Strategies for Empathetic Response Generationcited by 12
  32. 2025SWE-Bench-CL: Continual Learning for Coding Agentscited by 12
  33. 2024Instruction Following without Instruction Tuningcited by 11
  34. 2024ChemTEB: Chemical Text Embedding Benchmark, an Overview of Embedding Models Performance & Efficiency on a Specific Domaincited by 11
  35. 2025Syntriever: How to Train Your Retriever with Synthetic Data from LLMscited by 11
  36. 2025ModernBERT or DeBERTaV3? Examining Architecture and Data Influence on Transformer Encoder Models Performancecited by 11
  37. 2025InteractiveSurvey: An LLM-based Personalized and Interactive Survey Paper Generation Systemcited by 11
  38. 2025MIEB: Massive Image Embedding Benchmarkcited by 11

50 most cited of 255 All 255 (JSON)