Spacerini: Plug-and-play Search Engines with Pyserini and Hugging Face
Not on the map. Why?
Abstract (arXiv)
We present Spacerini, a tool that integrates the Pyserini toolkit for reproducible information retrieval research with Hugging Face to enable the seamless construction and deployment of interactive search engines. Spacerini makes state-of-the-art sparse and dense retrieval models more accessible to non-IR practitioners while minimizing deployment effort. This is useful for NLP researchers who want to better understand and validate their research by performing qualitative analyses of training corpora, for IR researchers who want to demonstrate new retrieval models integrated into the growing Pyserini ecosystem, and for third parties reproducing the work of other researchers. Spacerini is open source and includes utilities for loading, preprocessing, indexing, and deploying search engines locally and remotely. We demonstrate a portfolio of 13 search engines created with Spacerini for different use cases.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 17 of its 33 references that are in the map. First posted on arXiv 28 Feb 2023.
Builds on
17 of 33 references.
References · 17
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
- 2021Ethical and social risks of harm from Language Modelscited by 966
- 2021Datasets: A Community Library for Natural Language Processingcited by 530
- 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 270
- 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
- 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
- 2019Gradio: Hassle-Free Sharing and Testing of ML Models in the Wildcited by 178
- 2023Auditing large language models: a three-layered approachcited by 137
- 2019Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learningcited by 116
- 2022Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languagescited by 47
5 more
- 2022Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamicscited by 26
- 2016Shirtless and Dangerous: Quantifying Linguistic Signals of Gender Bias in an Online Fiction Writing Communitycited by 21
- 2020Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Datasetcited by 15
- 2022BigScience: A Case Study in the Social Construction of a Multilingual Large Language Modelcited by 11
- 2022Quote Erat Demonstrandum: A Web Interface for Exploring the Quotebank Corpuscited by 5
Built on by
Citing papers per year: 2023: 3.
3 papers.
- 2023StarCoder: may the source be with you!cited by 799
- 2023Read, Look or Listen? What's Needed for Solving a Multimodal Datasetcited by 2
- 2023Indicative Summarization of Long Discussionscited by 0
- 2023Indicative Summarization of Long Discussions3 Nov 2023
- 2023Read, Look or Listen? What's Needed for Solving a Multimodal Dataset6 Jul 2023
- 2023StarCoder: may the source be with you!9 May 2023