# A Unified Structured Query Understanding Framework for Industrial Semantic Search

arXiv: https://arxiv.org/abs/2605.27441 · HTML: https://mlsanity.com/paper/2605.27441/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 22 May 2026.

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [UltraFeedback: Boosting Language Models with Scaled AI Feedback](https://mlsanity.com/paper/2310.01377/) (2023)
- [Large Language Model based Long-tail Query Rewriting in Taobao Search](https://mlsanity.com/paper/2311.03758/) (2023)
- [GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulation](https://mlsanity.com/paper/2404.03746/) (2024)
- [QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation](https://mlsanity.com/paper/2210.15718/) (2022)
- [Query Routing for Homogeneous Tools: An Instantiation in the RAG Scenario](https://mlsanity.com/paper/2406.12429/) (2024)
- [Put Teacher in Student's Shoes: Cross-Distillation for Ultra-compact Model Compression Framework](https://mlsanity.com/paper/2507.04636/) (2025)
- [Exploring Query Understanding for Amazon Product Search](https://mlsanity.com/paper/2408.02215/) (2024)
- [Personalized Query Auto-Completion for Long and Short-Term Interests with Adaptive Detoxification Generation](https://mlsanity.com/paper/2505.20966/) (2025)
- [DeepTagger: Knowledge Enhanced Named Entity Recognition for Web-Based Ads Queries](https://mlsanity.com/paper/2306.17413/) (2023)
- [Natural language technology and query expansion: issues, state-of-the-art and perspectives](https://mlsanity.com/paper/2004.11093/) (2020)
- [Study of Workload Interference with Intelligent Routing on Dragonfly](https://mlsanity.com/paper/2403.16288/) (2024)
- [Powering Job Search at Scale: LLM-Enhanced Query Understanding in Job Matching Systems](https://mlsanity.com/paper/2509.09690/) (2025)
- [GREAT: Guiding Query Generation with a Trie for Recommending Related Search about Video at Kuaishou](https://mlsanity.com/paper/2507.15267/) (2025)
- [MixLM: High-Throughput and Effective LLM Ranking via Text-Embedding Mix-Interaction](https://mlsanity.com/paper/2512.07846/) (2025)

## Abstract (arXiv)

Query understanding in large-scale industrial search systems is typically implemented as a cascade of disparate, task-specific components. While individually optimizable, this fragmented architecture incurs high maintenance overhead and results in inconsistent behaviors, particularly for long-tail queries. In this work, we propose and deploy a unified structured query understanding system that consolidates these heterogeneous functions into a single Small Language Model (SLM) that performs schema-constrained generation. To address the data bottlenecks inherent in unified modeling, we introduce Query Illuminator, a dual-purpose framework serving as: (i) a teacher model for high-quality auto-annotation and distillation, and (ii) a surrogate judge for scalable evaluation where human labels are scarce. We validate this approach through extensive offline and online tests within LinkedIn's Job Search system. Furthermore, we demonstrate the framework's horizontal extensibility through a cross-domain case study on People Search. The results show improved user engagement and reduced operational costs, achieved while satisfying strict low-latency serving constraints on limited GPU resources.