mlsanityData to 7 Oct 2026

LotusFilter: Fast Diverse Nearest Neighbor Search via a Learned Cutoff Table

arXiv 2506.04790 ↗First posted 5 Jun 2025cs.CV cs.IR cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Approximate nearest neighbor search (ANNS) is an essential building block for applications like RAG but can sometimes yield results that are overly similar to each other. In certain scenarios, search results should be similar to the query and yet diverse. We propose LotusFilter, a post-processing module to diversify ANNS results. We precompute a cutoff table summarizing vectors that are close to each other. During the filtering, LotusFilter greedily looks up the table to delete redundant vectors from the candidates. We demonstrated that the LotusFilter operates fast (0.02 [ms/query]) in settings resembling real-world RAG applications, utilizing features such as OpenAI embeddings. Our code is publicly available at https://github.com/matsui528/lotf.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 25 of its 48 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

25 of 48 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  3. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  4. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  5. 2017Fine-tuning CNN Image Retrieval with No Human Annotationcited by 497
  6. 2018Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarkingcited by 189
  7. 2017The Case for Learned Index Structurescited by 188
  8. 2017Fast Approximate Nearest Neighbor Search With The Navigating Spreading-out Graphcited by 103
  9. 2021A Comprehensive Survey and Experimental Comparison of Graph-Based Approximate Nearest Neighbor Searchcited by 82
  10. 2019A Model for Learned Bloom Filters, and Optimizing by Sandwichingcited by 64
  11. 2022Results of the NeurIPS'21 Challenge on Billion-Scale Approximate Nearest Neighbor Searchcited by 45
  12. 2018Revisiting the Inverted Indices for Billion-Scale Approximate Nearest Neighborscited by 37
13 more
  1. 2019Learning Multi-dimensional Indexescited by 30
  2. 2020Tsunami: A Learned Multi-dimensional Index for Correlated Data and Skewed Workloadscited by 25
  3. 2024Results of the Big ANN: NeurIPS'23 competitioncited by 21
  4. 2018Quicker ADC : Unlocking the hidden potential of Product Quantization with SIMDcited by 14
  5. 2021Updatable Learned Index with Precise Positionscited by 13
  6. 2018Link and code: Fast indexing with graphs and compact regression codescited by 9
  7. 2023Fast Partitioned Learned Bloom Filtercited by 6
  8. 2020GLOW : Global Weighted Self-Attention Network for Web Searchcited by 4
  9. 2023Relative NN-Descent: A Fast Index Construction for Graph-Based Approximate Nearest Neighbor Searchcited by 4
  10. 2023General and Practical Tuning Method for Off-the-Shelf Graph-Based Index: SISAP Indexing Challenge Report by Team UTokyocited by 3
  11. 2024FlexFlood: Efficiently Updatable Learned Multi-dimensional Indexcited by 3
  12. 2022ARM 4-BIT PQ: SIMD-based Acceleration for Approximate Nearest Neighbor Search on ARMcited by 1
  13. 2024Theoretical and Empirical Analysis of Adaptive Entry Point Selection for Graph-based Approximate Nearest Neighbor Searchcited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025PLASMA: A Layout-Aware Benchmark Reveals Memory Layout Matters for Graph-based ANNS on GPUcited by 0