mlsanityData to 14 Oct 2026

EmbeddingGemma: Powerful and Lightweight Text Representations

arXiv 2509.20354 ↗First posted 24 Sep 2025cs.CL cs.AIcited by 139

Not on the map. Why?

Abstract (arXiv)

We introduce EmbeddingGemma, a new lightweight, open text embedding model based on the Gemma 3 language model family. Our innovative training recipe strategically captures knowledge from larger models via encoder-decoder initialization and geometric embedding distillation. We improve model robustness and expressiveness with a spread-out regularizer, and ensure generalizability by merging checkpoints from varied, optimized mixtures. Evaluated on the Massive Text Embedding Benchmark (MTEB) across multilingual, English, and code domains, EmbeddingGemma (300M) achieves state-of-the-art results. Notably, it outperforms prior top models, both proprietary and open, with fewer than 500M parameters, and provides performance comparable to models double its size, offering an exceptional performance-to-cost ratio. Remarkably, this lead persists when quantizing model weights or truncating embedding outputs. This makes EmbeddingGemma particularly well-suited for low-latency and high-throughput use cases such as on-device applications. We provide ablation studies exploring our key design choices. We release EmbeddingGemma to the community to promote further research.

Key facts. Cited by 139 papers in the map (data through 14 Oct 2026). Builds on 28 of its 32 references that are in the map. First posted on arXiv 24 Sep 2025.

Builds on

28 of 32 references.

References · 28

  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  2. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  3. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  4. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  5. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  6. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  7. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  8. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  9. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  10. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  11. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
  12. 2023Improving Text Embeddings with Large Language Modelscited by 303
16 more
  1. 2022Matryoshka Representation Learningcited by 282
  2. 2025Small Language Models are the Future of Agentic AIcited by 270
  3. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 228
  4. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 141
  5. 2024E5-V: Universal Embeddings with Multimodal Large Language Modelscited by 126
  6. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 125
  7. 2020XOR QA: Cross-lingual Open-Retrieval Question Answeringcited by 121
  8. 2025Gemini Embedding: Generalizable Embeddings from Geminicited by 94
  9. 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 84
  10. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 83
  11. 2025LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learningcited by 55
  12. 2024LongEmbed: Extending Embedding Models for Long Context Retrievalcited by 48
  13. 2017Learning Spread-out Local Feature Descriptorscited by 34
  14. 2023XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languagescited by 34
  15. 2025Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptationcited by 28
  16. 2025Adapting Decoder-Based Language Models for Diverse Encoder Downstream Taskscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 15; 2026: 124.

139 papers.

Sort:

By area:

  1. 2025VL-JEPA: Joint Embedding Predictive Architecture for Vision-languagecited by 68
  2. 2025RzenEmbed: Towards Comprehensive Multimodal Retrievalcited by 29
  3. 2025T5Gemma 2: Seeing, Reading, and Understanding Longercited by 17
  4. 2025ReasonEmbed: Enhanced Text Embeddings for Reasoning-Intensive Document Retrievalcited by 16
  5. 2026Action100M: A Large-scale Video Action Datasetcited by 12
  6. 2026Reasoning Models Generate Societies of Thoughtcited by 12
  7. 2026F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual Worldcited by 12
  8. 2026Scaling Behavior Cloning Improves Causal Reasoning: An Open Model for Real-Time Video Game Playingcited by 11
  9. 2025MindEval: Benchmarking Language Models on Multi-turn Mental Health Supportcited by 10
  10. 2026SciCoQA: Quality Assurance for Scientific Paper--Code Alignmentcited by 10
  11. 2026Diffusion-Pretrained Dense and Contextual Embeddingscited by 9
  12. 2026Semantic Search over 9 Million Mathematical Theoremscited by 8
38 more
  1. 2026Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memorycited by 8
  2. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  3. 2026DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Searchcited by 6
  4. 2025JEF-Hinter: Leveraging Offline Knowledge for Improving Web Agents Adaptationcited by 5
  5. 2025Search-R3: Unifying Reasoning and Embedding in Large Language Modelscited by 4
  6. 2026Reasoning models do not yet follow their reasoning in autonomous driving: The KITScenes LongTail Datasetcited by 4
  7. 2025PTEB: Towards Robust Text Embedding Evaluation via Stochastic Paraphrasing at Evaluation Time with LLMscited by 3
  8. 2025AfriMTEB and AfriE5: Benchmarking and Adapting Text Embedding Models for African Languagescited by 3
  9. 2025C2LLM Technical Report: A New Frontier in Code Retrieval via Adaptive Cross-Attention Poolingcited by 3
  10. 2026Towards Simulating Social Media Users with LLMs: Evaluating the Operational Validity of Conditioned Comment Predictioncited by 3
  11. 2026Efficient and High-Fidelity Omni Modality Retrievalcited by 3
  12. 2026MemSifter: Offloading LLM Memory Retrieval via Outcome-Driven Proxy Reasoningcited by 3
  13. 2026Do LLMs Know Tool Irrelevance? Demystifying Structural Alignment Bias in Tool Invocationscited by 3
  14. 2026Is Dimensionality a Barrier for Retrieval Models?cited by 3
  15. 2026OntoLearner: A Modular Python Library for Ontology Learning with Large Language Modelscited by 3
  16. 2026Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domaincited by 2
  17. 2026An Extreme Multi-label Text Classification (XMTC) Library Dataset: What if we took "Use of Practical AI in Digital Libraries" seriously?cited by 2
  18. 2026Zero-Shot Quantization via Weight-Space Arithmeticcited by 2
  19. 2026Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systemscited by 2
  20. 2026CanLegalRAGBench: Evaluating Retrieval-Augmented Generation on Canadian Case Lawcited by 2
  21. 2026On the Robustness of Multilingual Text Embedding Rankings Across Learning Tasks, Languages, and Benchmark Datasetscited by 2
  22. 2026MUNI: Multimodal Unified Latent Diffusion for Coherent Any-to-Any Generationcited by 2
  23. 2026Latent Goal Prediction from Language for Model-Based Planningcited by 2
  24. 2025Unified Work Embeddings: Contrastive Learning of a Bidirectional Multi-task Rankercited by 1
  25. 2025Cross-modal Retrieval Models for Stripped Binary Analysiscited by 1
  26. 2026HypRAG: Hyperbolic Dense Retrieval for Retrieval Augmented Generationcited by 1
  27. 2026SQuTR: A Robustness Benchmark for Spoken Query to Text Retrieval under Acoustic Noisecited by 1
  28. 2026HotelQuEST: Balancing Quality and Efficiency in Agentic Searchcited by 1
  29. 2026TrajPred: Trajectory-Conditioned Joint Embedding Prediction for Surgical Instrument-Tissue Interaction Recognition in Vision-Language Modelscited by 1
  30. 2026Incentives shape how humans co-create with generative AIcited by 1
  31. 2026Cross-Lingual Attention Distillation with Personality-Informed Generative Augmentation for Multilingual Personality Recognitioncited by 1
  32. 2026Predicting Post-Traumatic Epilepsy from Clinical Records using Large Language Model Embeddingscited by 1
  33. 2026Linear Semantic Segmentation for Low-Resource Spoken Dialectscited by 1
  34. 2026Differences in Text Generated by Diffusion and Autoregressive Language Modelscited by 1
  35. 2026BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Taskscited by 1
  36. 2026Energy-Efficient On-Device RAG on a Mobile NPU: System Design and Benchmark on Snapdragon X Elitecited by 1
  37. 2026CAPED: Context-Aware Privacy Exposure Defense for Mobile GUI Agentscited by 1
  38. 2026LEDGER: A Long-Context Benchmark of Corporate Annual Reports for Grounded Financial Retrieval and Extractioncited by 1

50 most cited of 139 All 139 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Recent advances in text embedding: A Comprehensive Review of Top-Performing Methods on the MTEB Benchmark
  2. 2026Giga-Embeddings: Mixture-of-Experts Encoders for High-Throughput Text Embeddings
  3. 2025Interpretable Embeddings with Sparse Autoencoders: A Data Analysis Toolkit
  4. 2025Layer by Layer: Uncovering Hidden Representations in Language Models
  5. 2026Scaling and Distilling Text Embeddings for Better Diffusibility
  6. 2025Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
  7. 2025Luxical: High-Speed Lexical-Dense Text Embeddings
  8. 2024Extracting Sentence Embeddings from Pretrained Transformer Models