# SitEmb-v1.5: Improved Context-Aware Dense Retrieval for Semantic Association and Long Story Comprehension arXiv: https://arxiv.org/abs/2508.01959 · HTML: https://mlsanity.com/paper/2508.01959/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. - Builds on 25 of its 30 references that are in the map. - First posted on arXiv 3 Aug 2025. ## Placed in - Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/) ## Builds on (most cited first, 25 of 25) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021) - [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020) - [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023) - [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022) - [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017) - [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024) - [∞Bench: Extending Long Context Evaluation Beyond 100K Tokens](https://mlsanity.com/paper/2402.13718/) (2024) - [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019) - [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023) - [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://mlsanity.com/paper/2402.01613/) (2024) - [CodeRAG-Bench: Can Retrieval Augment Code Generation?](https://mlsanity.com/paper/2406.14497/) (2024) - [NV-Retriever: Improving text embedding models with effective hard-negative mining](https://mlsanity.com/paper/2407.15831/) (2024) - [One Thousand and One Pairs: A "novel" challenge for long-context language models](https://mlsanity.com/paper/2406.16264/) (2024) - [KaLM-Embedding: Superior Training Data Brings A Stronger Embedding Model](https://mlsanity.com/paper/2501.01028/) (2025) - [Tevatron 2.0: Unified Document Retrieval Toolkit across Scale, Language, and Modality](https://mlsanity.com/paper/2505.02466/) (2025) - [DetectiveQA: Evaluating Long-Context Reasoning on Detective Novels](https://mlsanity.com/paper/2409.02465/) (2024) - [Personality Understanding of Fictional Characters during Book Reading](https://mlsanity.com/paper/2305.10156/) (2023) - [Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study](https://mlsanity.com/paper/2106.03826/) (2021) - [CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models](https://mlsanity.com/paper/2403.03514/) (2024) - [Fine-Grained Modeling of Narrative Context: A Coherence Perspective via Retrospective Questions](https://mlsanity.com/paper/2402.13551/) (2024) - [The Essence of Contextual Understanding in Theory of Mind: A Study on Question Answering with Story Characters](https://mlsanity.com/paper/2501.01705/) (2025) - [Plot Retrieval as an Assessment of Abstract Semantic Association](https://mlsanity.com/paper/2311.01666/) (2023) - [Previously on the Stories: Recap Snippet Identification for Story Reading](https://mlsanity.com/paper/2402.07271/) (2024) - [Ref-Long: Benchmarking the Long-context Referencing Capability of Long-context Language Models](https://mlsanity.com/paper/2507.09506/) (2025) ## Built on by (most cited first, 8 of 8) - [ComoRAG: A Cognitive-Inspired Memory-Organized RAG for Stateful Long Narrative Reasoning](https://mlsanity.com/paper/2508.10419/) (2025) - [AutoSurvey2: Empowering Researchers with Next Level Automated Literature Surveys](https://mlsanity.com/paper/2510.26012/) (2025) - [PRELUDE: A Benchmark Designed to Require Global Comprehension and Reasoning over Long Contexts](https://mlsanity.com/paper/2508.09848/) (2025) - [Query-focused and Memory-aware Reranker for Long Context Processing](https://mlsanity.com/paper/2602.12192/) (2026) - [Mindscape-Aware Retrieval Augmented Generation for Improved Long Context Understanding](https://mlsanity.com/paper/2512.17220/) (2025) - [Graph-Aware Late Chunking for Retrieval-Augmented Generation in Biomedical Literature](https://mlsanity.com/paper/2603.22633/) (2026) - [ExDR: Explanation-driven Dynamic Retrieval Enhancement for Multimodal Fake News Detection](https://mlsanity.com/paper/2601.15820/) (2026) - [InsightEmb: Learning Action-Intent Embeddings for Agentic Insight Retrieval](https://mlsanity.com/paper/2608.04761/) (2026) ## Abstract (arXiv) Retrieval-augmented generation (RAG) over long documents typically involves splitting the text into smaller chunks, which serve as the basic units for retrieval. However, due to dependencies across the original document, contextual information is often essential for accurately interpreting each chunk. To address this, prior work has explored encoding longer context windows to produce embeddings for longer chunks. Despite these efforts, gains in retrieval and downstream tasks remain limited. This is because (1) longer chunks strain the capacity of embedding models due to the increased amount of information they must encode, and (2) many real-world applications still require returning localized evidence due to constraints on model or human bandwidth. We propose an alternative approach to this challenge by representing short chunks in a way that is conditioned on a broader context window to enhance retrieval performance -- i.e., situating a chunk's meaning within its context. We further show that existing embedding models are not well-equipped to encode such situated context effectively, and thus introduce a new training paradigm and develop the situated embedding models (SitEmb). To evaluate our method, we curate a book-plot retrieval dataset specifically designed to assess situated retrieval capabilities. On this benchmark, our SitEmb-v1 model based on BGE-M3 substantially outperforms state-of-the-art embedding models, including several with up to 7-8B parameters, with only 1B parameters. Our 8B SitEmb-v1.5 model further improves performance by over 10% and shows strong results across different languages and several downstream applications.