# NeuralSearchX: Serving a Multi-billion-parameter Reranker for Multilingual Metasearch at a Low Cost

arXiv: https://arxiv.org/abs/2210.14837 · HTML: https://mlsanity.com/paper/2210.14837/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 39 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset](https://mlsanity.com/paper/2108.13897/) (2021)
- [Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations](https://mlsanity.com/paper/2102.10073/) (2021)
- [Pre-trained Language Model for Web-scale Retrieval in Baidu Search](https://mlsanity.com/paper/2106.03373/) (2021)
- [Cascade Ranking for Operational E-commerce Search](https://mlsanity.com/paper/1706.02093/) (2017)
- [No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrieval](https://mlsanity.com/paper/2206.02873/) (2022)
- [Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Dataset](https://mlsanity.com/paper/2007.07846/) (2020)
- [Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature](https://mlsanity.com/paper/2106.13375/) (2021)
- [Machine Learning on Volatile Instances](https://mlsanity.com/paper/2003.05649/) (2020)

## Built on by (most cited first, 1 of 1)

- [INACIA: Integrating Large Language Models in Brazilian Audit Courts: Opportunities and Challenges](https://mlsanity.com/paper/2401.05273/) (2024)

## Abstract (arXiv)

The widespread availability of search API's (both free and commercial) brings the promise of increased coverage and quality of search results for metasearch engines, while decreasing the maintenance costs of the crawling and indexing infrastructures. However, merging strategies frequently comprise complex pipelines that require careful tuning, which is often overlooked in the literature. In this work, we describe NeuralSearchX, a metasearch engine based on a multi-purpose large reranking model to merge results and highlight sentences. Due to the homogeneity of our architecture, we could focus our optimization efforts on a single component. We compare our system with Microsoft's Biomedical Search and show that our design choices led to a much cost-effective system with competitive QPS while having close to state-of-the-art results on a wide range of public benchmarks. Human evaluation on two domain-specific tasks shows that our retrieval system outperformed Google API by a large margin in terms of nDCG@10 scores. By describing our architecture and implementation in detail, we hope that the community will build on our design choices. The system is available at https://neuralsearchx.nsx.ai.