# Results of the Big ANN: NeurIPS'23 competition

arXiv: https://arxiv.org/abs/2409.17424 · HTML: https://mlsanity.com/paper/2409.17424/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Builds on (most cited first, 13 of 13)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [The Faiss library](https://mlsanity.com/paper/2401.08281/) (2024)
- [ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithms](https://mlsanity.com/paper/1807.05614/) (2018)
- [From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective](https://mlsanity.com/paper/2205.04733/) (2022)
- [Results of the NeurIPS'21 Challenge on Billion-Scale Approximate Nearest Neighbor Search](https://mlsanity.com/paper/2205.03763/) (2022)
- [FreshDiskANN: A Fast and Accurate Graph-Based ANN Index for Streaming Similarity Search](https://mlsanity.com/paper/2105.09613/) (2021)
- [Efficient Inverted Indexes for Approximate Retrieval over Learned Sparse Representations](https://mlsanity.com/paper/2404.18812/) (2024)
- [RoarGraph: A Projected Bipartite Graph for Efficient Cross-Modal Approximate Nearest Neighbor Search](https://mlsanity.com/paper/2408.08933/) (2024)
- [An Approximate Algorithm for Maximum Inner Product Search over Streaming Sparse Vectors](https://mlsanity.com/paper/2301.10622/) (2023)

## Built on by (most cited first, 22 of 22)

- [Inference-time sparse attention with asymmetric indexing](https://mlsanity.com/paper/2502.08246/) (2025)
- [MicroNN: An On-device Disk-resident Updatable Vector Database](https://mlsanity.com/paper/2504.05573/) (2025)
- [VIBE: Vector Index Benchmark for Embeddings](https://mlsanity.com/paper/2505.17810/) (2025)
- [Exploring Large Action Sets with Hyperspherical Embeddings using von Mises-Fisher Sampling](https://mlsanity.com/paper/2507.00518/) (2025)
- [Results and Retrospective Analysis of the CODS 2025 AssetOpsBench Challenge](https://mlsanity.com/paper/2605.08518/) (2026)
- [Machine learning and high dimensional vector search](https://mlsanity.com/paper/2502.16931/) (2025)
- [LotusFilter: Fast Diverse Nearest Neighbor Search via a Learned Cutoff Table](https://mlsanity.com/paper/2506.04790/) (2025)
- [Efficient Sketching and Nearest Neighbor Search Algorithms for Sparse Vector Sets](https://mlsanity.com/paper/2509.24815/) (2025)
- [Onyx: Cost-Efficient Disk-Oblivious ANN Search](https://mlsanity.com/paper/2604.20401/) (2026)
- [Scalable k-Means Clustering for Large k via Seeded Approximate Nearest-Neighbor Search](https://mlsanity.com/paper/2502.06163/) (2025)
- [MINT: Multi-Vector Search Index Tuning](https://mlsanity.com/paper/2504.20018/) (2025)
- [Distance Adaptive Beam Search for Provably Accurate Graph-Based Nearest Neighbor Search](https://mlsanity.com/paper/2505.15636/) (2025)
- [Passing the Baton: High Throughput Distributed Disk-Based Vector Search with BatANN](https://mlsanity.com/paper/2512.09331/) (2025)
- [Exqutor: Extended Query Optimizer for Vector-augmented Analytical Queries](https://mlsanity.com/paper/2512.09695/) (2025)
- [MCGI: Manifold-Consistent Graph Indexing for Billion-Scale Disk-Resident Vector Search](https://mlsanity.com/paper/2601.01930/) (2026)
- [Welfarist Formulations for Diverse Similarity Search](https://mlsanity.com/paper/2602.08742/) (2026)
- [Filtered Approximate Nearest Neighbor Search in Vector Databases: System Design and Performance Analysis](https://mlsanity.com/paper/2602.11443/) (2026)
- [Filtered ANN as a Phase Transition: When Selectivity-Estimation Error Causes Plan Regret](https://mlsanity.com/paper/2606.16341/) (2026)
- [ANNLib: A Development Framework for Efficient Approximate Nearest Neighbor Search](https://mlsanity.com/paper/2607.17582/) (2026)
- [VQ-bench: A Composable Vector Quantization Framework](https://mlsanity.com/paper/2608.11240/) (2026)
- [A Power Law in Logarithm's Clothing: On the Scalability of Graph-Based Vector Search](https://mlsanity.com/paper/2609.02143/) (2026)
- [Beyond Private Training: The New Landscape of AI Privacy](https://mlsanity.com/paper/2609.19456/) (2026)

## Abstract (arXiv)

The 2023 Big ANN Challenge, held at NeurIPS 2023, focused on advancing the state-of-the-art in indexing data structures and search algorithms for practical variants of Approximate Nearest Neighbor (ANN) search that reflect the growing complexity and diversity of workloads. Unlike prior challenges that emphasized scaling up classical ANN search ~citeDBLP:conf/nips/SimhadriWADBBCH21, this competition addressed filtered search, out-of-distribution data, sparse and streaming variants of ANNS. Participants developed and submitted innovative solutions that were evaluated on new standard datasets with constrained computational resources. The results showcased significant improvements in search accuracy and efficiency over industry-standard baselines, with notable contributions from both academic and industrial teams. This paper summarizes the competition tracks, datasets, evaluation metrics, and the innovative approaches of the top-performing submissions, providing insights into the current advancements and future directions in the field of approximate nearest neighbor search.