QUPID: Quantified Understanding for Enhanced Performance, Insights, and Decisions in Korean Search Engines
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) have been widely used for relevance assessment in information retrieval. However, our study demonstrates that combining two distinct small language models (SLMs) with different architectures can outperform LLMs in this task. Our approach -- QUPID -- integrates a generative SLM with an embedding-based SLM, achieving higher relevance judgment accuracy while reducing computational costs compared to state-of-the-art LLM solutions. This computational efficiency makes QUPID highly scalable for real-world search systems processing millions of queries daily. In experiments across diverse document types, our method demonstrated consistent performance improvements (Cohen's Kappa of 0.646 versus 0.387 for leading LLMs) while offering 60x faster inference times. Furthermore, when integrated into production search pipelines, QUPID improved nDCG@5 scores by 1.9%. These findings underscore how architectural diversity in model combinations can significantly enhance both search relevance and operational efficiency in information retrieval systems.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 43 references that are in the map. First posted on arXiv 12 May 2025.
Builds on
27 of 43 references.
References · 27
- 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
- 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
- 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
- 2023Large Language Models for Information Retrieval: A Surveycited by 331
- 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 272
- 2019Document Expansion by Query Predictioncited by 231
- 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 165
- 2023Large language models can accurately predict searcher preferencescited by 131
- 2020The Cost of Training NLP Models: A Concise Overviewcited by 98
- 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
- 2023Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labelscited by 86
- 2023Perspectives on Large Language Models for Relevance Judgmentcited by 85
15 more
- 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 84
- 2023Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrievalcited by 78
- 2021CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learningcited by 68
- 2023Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Promptscited by 45
- 2024UMBRELA: UMbrela is the (Open-Source Reproduction of the) Bing RELevance Assessorcited by 42
- 2018Personalized Language Model for Query Auto-Completioncited by 25
- 2024SLMRec: Distilling Large Language Models into Small for Sequential Recommendationcited by 18
- 2024R-Bot: An LLM-based Query Rewrite Systemcited by 16
- 2024Leveraging small language models for Text2SPARQL tasks to improve the resilience of AI assistancecited by 13
- 2024Performance of Recent Large Language Models for a Low-Resourced Languagecited by 7
- 2019Subword Language Model for Query Auto-Completioncited by 4
- 2020Efficient and Effective Query Auto-Completioncited by 4
- 2023Making Small Language Models Better Multi-task Learners with Mixture-of-Task-Adapterscited by 3
- 2024Best in Tau@LLMJudge: Criteria-Based Relevance Evaluation with Llama3cited by 3
- 2023Offline Pseudo Relevance Feedback for Efficient and Effective Single-pass Dense Retrievalcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.