# Gemini Embedding: Generalizable Embeddings from Gemini

arXiv: https://arxiv.org/abs/2503.07891 · HTML: https://mlsanity.com/paper/2503.07891/

## Key facts

- Cited by 94 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 41 references that are in the map.
- First posted on arXiv 10 Mar 2025.

## Builds on (most cited first, 30 of 34)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [Text and Code Embeddings by Contrastive Pre-Training](https://mlsanity.com/paper/2201.10005/) (2022)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [Matryoshka Representation Learning](https://mlsanity.com/paper/2205.13147/) (2022)
- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://mlsanity.com/paper/2209.11755/) (2022)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks](https://mlsanity.com/paper/2410.05160/) (2024)
- [MMTEB: Massive Multilingual Text Embedding Benchmark](https://mlsanity.com/paper/2502.13595/) (2025)
- [XOR QA: Cross-lingual Open-Retrieval Question Answering](https://mlsanity.com/paper/2010.11856/) (2020)
- [Gecko: Versatile Text Embeddings Distilled from Large Language Models](https://mlsanity.com/paper/2403.20327/) (2024)
- [Making Text Embedders Few-Shot Learners](https://mlsanity.com/paper/2409.15700/) (2024)
- [InPars-v2: Large Language Models as Efficient Dataset Generators for Information Retrieval](https://mlsanity.com/paper/2301.01820/) (2023)
- [XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages](https://mlsanity.com/paper/2305.11938/) (2023)

## Built on by (most cited first, 30 of 94)

- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory](https://mlsanity.com/paper/2509.25140/) (2025)
- [Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking](https://mlsanity.com/paper/2601.04720/) (2026)
- [EmbeddingGemma: Powerful and Lightweight Text Representations](https://mlsanity.com/paper/2509.20354/) (2025)
- [On the Theoretical Limitations of Embedding-Based Retrieval](https://mlsanity.com/paper/2508.21038/) (2025)
- [Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks](https://mlsanity.com/paper/2511.07025/) (2025)
- [KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model](https://mlsanity.com/paper/2506.20923/) (2025)
- [The Geometry of Reasoning: Flowing Logics in Representation Space](https://mlsanity.com/paper/2510.09782/) (2025)
- [SweRank: Software Issue Localization with Code Ranking](https://mlsanity.com/paper/2505.07849/) (2025)
- [Should We Still Pretrain Encoders with Masked Language Modeling?](https://mlsanity.com/paper/2507.00994/) (2025)
- [Efficient Code Embeddings from Code Generation Models](https://mlsanity.com/paper/2508.21290/) (2025)
- [Coding Agents are Effective Long-Context Processors](https://mlsanity.com/paper/2603.20432/) (2026)
- [SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Model](https://mlsanity.com/paper/2510.12709/) (2025)
- [Nemotron ColEmbed V2: Top-Performing Late Interaction Embedding Models for Visual Document Retrieval](https://mlsanity.com/paper/2602.03992/) (2026)
- [Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini](https://mlsanity.com/paper/2605.27295/) (2026)
- [Large Reasoning Embedding Models: Towards Next-Generation Dense Retrieval Paradigm](https://mlsanity.com/paper/2510.14321/) (2025)
- [Massive Sound Embedding Benchmark (MSEB)](https://mlsanity.com/paper/2602.07143/) (2026)
- [Diffusion-Pretrained Dense and Contextual Embeddings](https://mlsanity.com/paper/2602.11151/) (2026)
- [Extracting alignment data in open models](https://mlsanity.com/paper/2510.18554/) (2025)
- [OBLIQ-Bench: Exposing Overlooked Bottlenecks in Modern Retrievers with Latent and Implicit Queries](https://mlsanity.com/paper/2605.06235/) (2026)
- [Advancing Semantic Caching for LLMs with Domain-Specific Embeddings and Synthetic Data](https://mlsanity.com/paper/2504.02268/) (2025)
- [WAVE: Learning Unified & Versatile Audio-Visual Embeddings with Multimodal LLM](https://mlsanity.com/paper/2509.21990/) (2025)
- [DS-STAR: Data Science Agent for Solving Diverse Tasks across Heterogeneous Formats and Open-Ended Queries](https://mlsanity.com/paper/2509.21825/) (2025)
- [Persona Generators: Generating Diverse Synthetic Personas for Arbitrary Contexts](https://mlsanity.com/paper/2602.03545/) (2026)
- [CSRv2: Unlocking Ultra-Sparse Embeddings](https://mlsanity.com/paper/2602.05735/) (2026)
- [Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy](https://mlsanity.com/paper/2603.05637/) (2026)
- [Learning Retrieval Models with Sparse Autoencoders](https://mlsanity.com/paper/2603.13277/) (2026)
- [The Other Mind: How Language Models Exhibit Human Temporal Cognition](https://mlsanity.com/paper/2507.15851/) (2025)
- [From Topology to Retrieval: Decoding Embedding Spaces with Unified Signatures](https://mlsanity.com/paper/2511.22150/) (2025)
- [The Embedder's Dilemma: LLMs Are Better, but at What Cost?](https://mlsanity.com/paper/2608.12875/) (2026)

## Abstract (arXiv)

In this report, we introduce Gemini Embedding, a state-of-the-art embedding model leveraging the power of Gemini, Google's most capable large language model. Capitalizing on Gemini's inherent multilingual and code understanding capabilities, Gemini Embedding produces highly generalizable embeddings for text spanning numerous languages and textual modalities. The representations generated by Gemini Embedding can be precomputed and applied to a variety of downstream tasks including classification, similarity, clustering, ranking, and retrieval. Evaluated on the Massive Multilingual Text Embedding Benchmark (MMTEB), which includes over one hundred tasks across 250+ languages, Gemini Embedding substantially outperforms prior state-of-the-art models, demonstrating considerable improvements in embedding quality. Achieving state-of-the-art performance across MMTEB's multilingual, English, and code benchmarks, our unified model demonstrates strong capabilities across a broad selection of tasks and surpasses specialized domain-specific models.