# Interpret and Control Dense Retrieval with Sparse Latent Features

arXiv: https://arxiv.org/abs/2411.00786 · HTML: https://mlsanity.com/paper/2411.00786/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2024.

## Built on by (most cited first, 10 of 10)

- [Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval](https://mlsanity.com/paper/2506.00041/) (2025)
- [A Survey on Retrieval And Structuring Augmented Generation with Large Language Models](https://mlsanity.com/paper/2509.10697/) (2025)
- [Learning Retrieval Models with Sparse Autoencoders](https://mlsanity.com/paper/2603.13277/) (2026)
- [IterKey: Iterative Keyword Generation with LLMs for Enhanced Retrieval Augmented Generation](https://mlsanity.com/paper/2505.08450/) (2025)
- [IMRNNs: An Efficient Method for Interpretable Dense Retrieval via Embedding Modulation](https://mlsanity.com/paper/2601.20084/) (2026)
- [DEO: Training-Free Direct Embedding Optimization for Negation-Aware Retrieval](https://mlsanity.com/paper/2603.09185/) (2026)
- [Xetrieval: Mechanistically Explaining Dense Retrieval](https://mlsanity.com/paper/2605.29507/) (2026)
- [From Tokens to Concepts: Leveraging SAE for SPLADE](https://mlsanity.com/paper/2604.21511/) (2026)
- [Beyond Token Scale: Chunk-Level Sparse Autoencoders for Reliable Semantic Feature Discovery](https://mlsanity.com/paper/2609.35521/) (2026)
- [Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility](https://mlsanity.com/paper/2610.20139/) (2026)

## Abstract (arXiv)

Dense embeddings deliver strong retrieval performance but often lack interpretability and controllability. This paper introduces a novel approach using sparse autoencoders (SAE) to interpret and control dense embeddings via the learned latent sparse features. Our key contribution is the development of a retrieval-oriented contrastive loss, which ensures the sparse latent features remain effective for retrieval tasks and thus meaningful to interpret. Experimental results demonstrate that both the learned latent sparse features and their reconstructed embeddings retain nearly the same retrieval accuracy as the original dense vectors, affirming their faithfulness. Our further examination of the sparse latent space reveals interesting features underlying the dense embeddings and we can control the retrieval behaviors via manipulating the latent sparse features, for example, prioritizing documents from specific perspectives in the retrieval results.