mlsanityData to 7 Oct 2026

SearchGym: A Modular Infrastructure for Cross-Platform Benchmarking and Hybrid Search Orchestration

arXiv 2603.04402 ↗First posted 2 Jan 2026cs.IR cs.CLcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

The rapid growth of Retrieval-Augmented Generation (RAG) has created a proliferation of toolkits, yet a fundamental gap remains between experimental prototypes and robust, production-ready systems. We present SearchGym, a modular infrastructure designed for cross-platform benchmarking and hybrid search orchestration. Unlike existing model-centric frameworks, SearchGym decouples data representation, embedding strategies, and retrieval logic into stateful abstractions: Dataset, VectorSet, and App. This separation enables a Compositional Config Algebra, allowing designers to synthesize entire systems from hierarchical configurations while ensuring perfect reproducibility. Moreover, we analyze the "Top-k Cognizance" in hybrid retrieval pipelines, demonstrating that the optimal sequence of semantic ranking and structured filtering is highly dependent on filter strength. Evaluated on the LitSearch expert-annotated benchmark, SearchGym achieves a 70% Top-100 retrieval rate. SearchGym reveals a design tension between generalizability and optimizability, presenting the potential where engineering optimization may serve as a tool for uncovering the causal mechanisms inherent in information retrieval across heterogeneous domains. An open-source implementation of SearchGym is available at: https://github.com/JeromeTH/search-gym

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 5 of its 11 references that are in the map. First posted on arXiv 2 Jan 2026.

Builds on

5 of 11 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Survey18 Dec

Other references · 4

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  3. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  4. 2024LitSearch: A Retrieval Benchmark for Scientific Literature Searchcited by 47

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Balancing the Blend: An Experimental Analysis of Trade-offs in Hybrid Search
  2. 2026HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditions
  3. 2025Benchmarking Deep Search over Heterogeneous Enterprise Data
  4. 2026RAISE: RAG Design as an Architecture Search Problem
  5. 2025RAGSmith: A Framework for Finding the Optimal Composition of Retrieval-Augmented Generation Methods Across Datasets
  6. 2025LevelRAG: Enhancing Retrieval-Augmented Generation with Multi-hop Logic Planning over Rewriting Augmented Searchers
  7. 2024Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval-Augmented Large Language Models
  8. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning