# QUPID: Quantified Understanding for Enhanced Performance, Insights, and Decisions in Korean Search Engines

arXiv: https://arxiv.org/abs/2505.07345 · HTML: https://mlsanity.com/paper/2505.07345/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 12 May 2025.

## Builds on (most cited first, 27 of 27)

- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [Improving Passage Retrieval with Zero-Shot Question Generation](https://mlsanity.com/paper/2204.07496/) (2022)
- [Large language models can accurately predict searcher preferences](https://mlsanity.com/paper/2309.10621/) (2023)
- [The Cost of Training NLP Models: A Concise Overview](https://mlsanity.com/paper/2004.08900/) (2020)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labels](https://mlsanity.com/paper/2310.14122/) (2023)
- [Perspectives on Large Language Models for Relevance Judgment](https://mlsanity.com/paper/2304.09161/) (2023)
- [NV-Retriever: Improving text embedding models with effective hard-negative mining](https://mlsanity.com/paper/2407.15831/) (2024)
- [Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrieval](https://mlsanity.com/paper/2312.15503/) (2023)
- [CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning](https://mlsanity.com/paper/2112.08558/) (2021)
- [Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts](https://mlsanity.com/paper/2306.11372/) (2023)
- [UMBRELA: UMbrela is the (Open-Source Reproduction of the) Bing RELevance Assessor](https://mlsanity.com/paper/2406.06519/) (2024)
- [Personalized Language Model for Query Auto-Completion](https://mlsanity.com/paper/1804.09661/) (2018)
- [SLMRec: Distilling Large Language Models into Small for Sequential Recommendation](https://mlsanity.com/paper/2405.17890/) (2024)
- [R-Bot: An LLM-based Query Rewrite System](https://mlsanity.com/paper/2412.01661/) (2024)
- [Leveraging small language models for Text2SPARQL tasks to improve the resilience of AI assistance](https://mlsanity.com/paper/2405.17076/) (2024)
- [Performance of Recent Large Language Models for a Low-Resourced Language](https://mlsanity.com/paper/2407.21330/) (2024)
- [Subword Language Model for Query Auto-Completion](https://mlsanity.com/paper/1909.00599/) (2019)
- [Efficient and Effective Query Auto-Completion](https://mlsanity.com/paper/2005.06213/) (2020)
- [Making Small Language Models Better Multi-task Learners with Mixture-of-Task-Adapters](https://mlsanity.com/paper/2309.11042/) (2023)
- [Best in Tau@LLMJudge: Criteria-Based Relevance Evaluation with Llama3](https://mlsanity.com/paper/2410.14044/) (2024)
- [Offline Pseudo Relevance Feedback for Efficient and Effective Single-pass Dense Retrieval](https://mlsanity.com/paper/2308.10191/) (2023)

## Built on by (most cited first, 1 of 1)

- [From Relevance to Authority: Authority-aware Generative Retrieval in Web Search Engines](https://mlsanity.com/paper/2604.13468/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have been widely used for relevance assessment in information retrieval. However, our study demonstrates that combining two distinct small language models (SLMs) with different architectures can outperform LLMs in this task. Our approach -- QUPID -- integrates a generative SLM with an embedding-based SLM, achieving higher relevance judgment accuracy while reducing computational costs compared to state-of-the-art LLM solutions. This computational efficiency makes QUPID highly scalable for real-world search systems processing millions of queries daily. In experiments across diverse document types, our method demonstrated consistent performance improvements (Cohen's Kappa of 0.646 versus 0.387 for leading LLMs) while offering 60x faster inference times. Furthermore, when integrated into production search pipelines, QUPID improved nDCG@5 scores by 1.9%. These findings underscore how architectural diversity in model combinations can significantly enhance both search relevance and operational efficiency in information retrieval systems.