# Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks

arXiv: https://arxiv.org/abs/2511.07025 · HTML: https://mlsanity.com/paper/2511.07025/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 48 references that are in the map.
- First posted on arXiv 10 Nov 2025.

## Builds on (most cited first, 28 of 28)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Multilingual E5 Text Embeddings: A Technical Report](https://mlsanity.com/paper/2402.05672/) (2024)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification](https://mlsanity.com/paper/2011.03088/) (2020)
- [PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them](https://mlsanity.com/paper/2102.07033/) (2021)
- [MAmmoTH2: Scaling Instructions from the Web](https://mlsanity.com/paper/2405.03548/) (2024)
- [Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval](https://mlsanity.com/paper/2108.08787/) (2021)
- [SciFact-Open: Towards open-domain scientific claim verification](https://mlsanity.com/paper/2210.13777/) (2022)
- [Gemini Embedding: Generalizable Embeddings from Gemini](https://mlsanity.com/paper/2503.07891/) (2025)
- [NV-Retriever: Improving text embedding models with effective hard-negative mining](https://mlsanity.com/paper/2407.15831/) (2024)
- [GooAQ: Open Question Answering with Diverse Answer Types](https://mlsanity.com/paper/2104.08727/) (2021)
- [I Wish I Would Have Loved This One, But I Didn't -- A Multilingual Dataset for Counterfactual Detection in Product Reviews](https://mlsanity.com/paper/2104.06893/) (2021)
- [TWEAC: Transformer with Extendable QA Agent Classifiers](https://mlsanity.com/paper/2104.07081/) (2021)
- [Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Model](https://mlsanity.com/paper/2507.05513/) (2025)
- [Multi-granular Legal Topic Classification on Greek Legislation](https://mlsanity.com/paper/2109.15298/) (2021)
- [SamToNe: Improving Contrastive Loss for Dual Encoder Retrieval Models with Same Tower Negatives](https://mlsanity.com/paper/2306.02516/) (2023)
- [Towards Better Instruction Following Retrieval Models](https://mlsanity.com/paper/2505.21439/) (2025)

## Built on by (most cited first, 30 of 51)

- [Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking](https://mlsanity.com/paper/2601.04720/) (2026)
- [Nemotron ColEmbed V2: Top-Performing Late Interaction Embedding Models for Visual Document Retrieval](https://mlsanity.com/paper/2602.03992/) (2026)
- [Soft Contamination Means Benchmarks Test Shallow Generalization](https://mlsanity.com/paper/2602.12413/) (2026)
- [Query-focused and Memory-aware Reranker for Long Context Processing](https://mlsanity.com/paper/2602.12192/) (2026)
- [BidirLM: From Text to Omnimodal Bidirectional Encoders by Adapting and Composing Causal LLMs](https://mlsanity.com/paper/2604.02045/) (2026)
- [The Embedder's Dilemma: LLMs Are Better, but at What Cost?](https://mlsanity.com/paper/2608.12875/) (2026)
- [On Conditional Stochastic Interpolation for Generative Nonlinear Sufficient Dimension Reduction](https://mlsanity.com/paper/2512.18971/) (2025)
- [FaithLens: Detecting and Explaining Faithfulness Hallucination](https://mlsanity.com/paper/2512.20182/) (2025)
- [Simple LLM Baselines are Competitive for Model Diffing](https://mlsanity.com/paper/2602.10371/) (2026)
- [How Do Decoder-Only LLMs Perceive Users? Rethinking Attention Masking for User Representation Learning](https://mlsanity.com/paper/2602.10622/) (2026)
- [SOTAlign: Semi-Supervised Alignment of Unimodal Vision and Language Models via Optimal Transport](https://mlsanity.com/paper/2602.23353/) (2026)
- [Does My Embedding Reflect That A = B? Evaluating Mathematical Equivalence in Embedding Models](https://mlsanity.com/paper/2606.23959/) (2026)
- [PosIR: Position-Aware Heterogeneous Information Retrieval Benchmark](https://mlsanity.com/paper/2601.08363/) (2026)
- [On the Robustness of Multilingual Text Embedding Rankings Across Learning Tasks, Languages, and Benchmark Datasets](https://mlsanity.com/paper/2605.31142/) (2026)
- [What LLMs Think When You Don't Tell Them What to Think About?](https://mlsanity.com/paper/2602.01689/) (2026)
- [Query as Anchor: Scenario-Adaptive User Representation via Large Language Model](https://mlsanity.com/paper/2602.14492/) (2026)
- [Polyglot Teachers: Evaluating Language Models for Multilingual Synthetic Data Generation](https://mlsanity.com/paper/2604.11290/) (2026)
- [Predicting Post-Traumatic Epilepsy from Clinical Records using Large Language Model Embeddings](https://mlsanity.com/paper/2604.14547/) (2026)
- [Document-as-Image Representations Fall Short for Scientific Retrieval](https://mlsanity.com/paper/2604.18508/) (2026)
- [The Harder Text Embedding Benchmark (HTEB): Beyond One-dimensional Static Robustness](https://mlsanity.com/paper/2605.28190/) (2026)
- [No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval](https://mlsanity.com/paper/2605.30120/) (2026)
- [Consistent and Distinctive: LLM Benchmark Efficiency via Maximum Independent Set Prompt Selection on Similarity Graphs](https://mlsanity.com/paper/2606.01400/) (2026)
- [SHIFT: Semantic Harmonization via Index-side Feature Transformation for Multilingual Information Retrieval](https://mlsanity.com/paper/2606.18801/) (2026)
- [Formalizing Latent Thoughts: Four Axioms of Thought Representation in LLMs](https://mlsanity.com/paper/2606.27378/) (2026)
- [SABER-Math: Automated Benchmark for Information Retrieval Evaluation in Mathematics](https://mlsanity.com/paper/2606.29894/) (2026)
- [STELLA: Self-Reflective Terminology-Aware Framework for Building an Aerospace Information Retrieval Benchmark](https://mlsanity.com/paper/2601.03496/) (2026)
- [Are Emotions Arranged in a Circle? Geometric Analysis of Emotion Representations via Hyperspherical Contrastive Learning](https://mlsanity.com/paper/2601.06575/) (2026)
- [Modular Expert Merging for Biomedical Retrieval](https://mlsanity.com/paper/2602.04731/) (2026)
- [BLUEPRINT Rebuilding a Legacy: Multimodal Retrieval for Complex Engineering Drawings and Documents](https://mlsanity.com/paper/2602.13345/) (2026)
- [GraphER: An Efficient Graph-Based Enrichment and Reranking Method for Retrieval-Augmented Generation](https://mlsanity.com/paper/2603.24925/) (2026)

## Abstract (arXiv)

We introduce llama-embed-nemotron-8b, an open-weights text embedding model that achieves state-of-the-art performance on the Multilingual Massive Text Embedding Benchmark (MMTEB) leaderboard as of October 21, 2025. While recent models show strong performance, their training data or methodologies are often not fully disclosed. We aim to address this by developing a fully open-source model, publicly releasing its weights and detailed ablation studies, and planning to share the curated training datasets. Our model demonstrates superior performance across all major embedding tasks -- including retrieval, classification and semantic textual similarity (STS) -- and excels in challenging multilingual scenarios, such as low-resource languages and cross-lingual setups. This state-of-the-art performance is driven by a novel data mix of 16.1 million query-document pairs, split between 7.7 million samples from public datasets and 8.4 million synthetically generated examples from various open-weight LLMs. One of our key contributions is a detailed ablation study analyzing core design choices, including a comparison of contrastive loss implementations, an evaluation of synthetic data generation (SDG) strategies, and the impact of model merging. The llama-embed-nemotron-8b is an instruction-aware model, supporting user-defined instructions to enhance performance for specific use-cases. This combination of top-tier performance, broad applicability, and user-driven flexibility enables it to serve as a universal text embedding solution.