mlsanityData to 7 Oct 2026

Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents

arXiv 2310.19923 ↗First posted 30 Oct 2023cs.CL cs.AI cs.LGcited by 103

Not on the map. Why?

Abstract (arXiv)

Text embedding models have emerged as powerful tools for transforming sentences into fixed-sized feature vectors that encapsulate semantic information. While these models are essential for tasks like information retrieval, semantic clustering, and text re-ranking, most existing open-source models, especially those built on architectures like BERT, struggle to represent lengthy documents and often resort to truncation. One common approach to mitigate this challenge involves splitting documents into smaller paragraphs for embedding. However, this strategy results in a much larger set of vectors, consequently leading to increased memory consumption and computationally intensive vector searches with elevated latency. To address these challenges, we introduce Jina Embeddings 2, an open-source text embedding model capable of accommodating up to 8192 tokens. This model is designed to transcend the conventional 512-token limit and adeptly process long documents. Jina Embeddings 2 not only achieves state-of-the-art performance on a range of embedding-related tasks in the MTEB benchmark but also matches the performance of OpenAI's proprietary ada-002 model. Additionally, our experiments indicate that an extended context can enhance performance in tasks such as NarrativeQA.

Key facts. Cited by 103 papers in the map (data through 7 Oct 2026). Builds on 28 of its 36 references that are in the map. First posted on arXiv 30 Oct 2023.

Builds on

28 of 36 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  9. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  10. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  11. 2020GLU Variants Improve Transformercited by 1,724
  12. 2017Decoupled Weight Decay Regularizationcited by 1,722
16 more
  1. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  2. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  3. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  4. 2017Mixed Precision Trainingcited by 1,135
  5. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  6. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  7. 2016Language Modeling with Gated Convolutional Networkscited by 688
  8. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  9. 2020Understanding the Behaviour of Contrastive Losscited by 397
  10. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  11. 2018Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Taskscited by 214
  12. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  13. 2019BIGPATENT: A Large-Scale Dataset for Abstractive and Coherent Summarizationcited by 155
  14. 2019Transformers without Tears: Improving the Normalization of Self-Attentioncited by 153
  15. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  16. 2023Jina Embeddings: A Novel Set of High-Performance Sentence Embedding Modelscited by 35

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 37; 2025: 41; 2026: 24.

103 papers.

Sort:

By area:

  1. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  2. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
  3. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 233
  4. 2024Generative Representational Instruction Tuningcited by 227
  5. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 138
  6. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
  7. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 89
  8. 2024Searching for Best Practices in Retrieval-Augmented Generationcited by 83
  9. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 72
  10. 2024Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Modelscited by 70
  11. 2024TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Modelscited by 63
  12. 2024Arctic-Embed 2.0: Multilingual Retrieval Without Compromisecited by 57
38 more
  1. 2024ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilitiescited by 52
  2. 2024LongEmbed: Extending Embedding Models for Long Context Retrievalcited by 47
  3. 2024Jina CLIP: Your CLIP Model Is Also Your Text Retrievercited by 46
  4. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32
  5. 2024Jina-ColBERT-v2: A General-Purpose Multilingual Late Interaction Retrievercited by 25
  6. 2024Balanced Data Sampling for Language Model Training with Clusteringcited by 22
  7. 2024Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarioscited by 22
  8. 2025SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugscited by 22
  9. 2024Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddingscited by 20
  10. 2024Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Modelscited by 20
  11. 2024MADial-Bench: Towards Real-world Evaluation of Memory-Augmented Dialogue Generationcited by 15
  12. 2025FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documentscited by 15
  13. 2025Give LLMs a Security Course: Securing Retrieval-Augmented Code Generation via Knowledge Injectioncited by 15
  14. 2025DocLens : A Tool-Augmented Multi-Agent Framework for Long Visual Document Understandingcited by 15
  15. 2024AIR-Bench: Automated Heterogeneous Information Retrieval Benchmarkcited by 12
  16. 2025Reconstructing Context: Evaluating Advanced Chunking Strategies for Retrieval-Augmented Generationcited by 12
  17. 2025Efficient Code Embeddings from Code Generation Modelscited by 12
  18. 2025Retrieval-of-Thought: Efficient Reasoning via Reusing Thoughtscited by 12
  19. 2024LLM2CLIP: Powerful Language Model Unlocks Richer Cross-Modality Representationcited by 11
  20. 2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
  21. 2025SAIL-Embedding Technical Report: Omni-modal Embedding Foundation Modelcited by 10
  22. 2024Length-Induced Embedding Collapse in PLM-based Modelscited by 9
  23. 2026Diffusion-Pretrained Dense and Contextual Embeddingscited by 9
  24. 2024Generalist embedding models are better at short-context clinical semantic search than specialized embedding modelscited by 7
  25. 2024Unveiling the Secrets of Engaging Conversations: Factors that Keep Users Hooked on Role-Playing Dialog Agentscited by 7
  26. 2025RTLRepoCoder: Repository-Level RTL Code Completion through the Combination of Fine-Tuning and Retrieval Augmentationcited by 7
  27. 2025Exploratory Retrieval-Augmented Planning For Continual Embodied Instruction Followingcited by 7
  28. 2025HeteRAG: A Heterogeneous Retrieval-augmented Generation Framework with Decoupled Knowledge Representationscited by 6
  29. 2025VIBE: Vector Index Benchmark for Embeddingscited by 6
  30. 2025SCOPE: A Generative Approach for LLM Prompt Compressioncited by 6
  31. 2025ToolDreamer: Instilling LLM Reasoning Into Tool Retrieverscited by 6
  32. 2024Advancing Semantic Textual Similarity Modeling: A Regression Framework with Translated ReLU and Smooth K2 Losscited by 5
  33. 2024Pcc-tuning: Breaking the Contrastive Learning Ceiling in Semantic Textual Similaritycited by 5
  34. 2025Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Modelscited by 5
  35. 2025Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Modelscited by 5
  36. 2025GasAgent: A Multi-Agent Framework for Automated Gas Optimization in Smart Contractscited by 5
  37. 2026LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculumcited by 5
  38. 2025Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoningcited by 4

50 most cited of 103 All 103 (JSON)