Semantic Search At LinkedIn
Not on the map. Why?
Abstract (arXiv)
Semantic search with large language models (LLMs) enables retrieval by meaning rather than keyword overlap, but scaling it requires major inference efficiency advances. We present LinkedIn's LLM-based semantic search framework for AI Job Search and AI People Search, combining an LLM relevance judge, embedding-based retrieval, and a compact Small Language Model trained via multi-teacher distillation to jointly optimize relevance and engagement. A prefill-oriented inference architecture co-designed with model pruning, context compression, and text-embedding hybrid interactions boosts ranking throughput by over 75x under a fixed latency constraint while preserving near-teacher-level NDCG, enabling one of the first production LLM-based ranking systems with efficiency comparable to traditional approaches and delivering significant gains in quality and user engagement.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 19 of its 46 references that are in the map. First posted on arXiv 7 Feb 2026.
Builds on
19 of 46 references.
References · 19
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
- 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
- 2023A Survey on Large Language Models for Recommendationcited by 378
- 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 336
- 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
- 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 143
- 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 138
- 2024Liger Kernel: Efficient Triton Kernels for LLM Trainingcited by 123
- 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 83
- 2023ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendationcited by 76
7 more
- 2024HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modelingcited by 63
- 2023A Two-Stage Adaptation of Large Language Models for Text Rankingcited by 17
- 2024OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimizationcited by 17
- 2020Understanding BERT Rankers Under Distillationcited by 15
- 2023Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start Recommendationscited by 15
- 2025From Reasoning LLMs to BERT: A Two-Stage Distillation Framework for Search Relevancecited by 7
- 2025Scaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Searchcited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Hierarchical Long-Term Semantic Memory for LinkedIn's Hiring Agent
- 2026PRECISE: Reducing the Bias of LLM Evaluations Using Prediction-Powered Ranking Estimation
- 2024Enhancing Cloud-Based Large Language Model Processing with Elasticsearch and Transformer Models
- 2025Orchestrating Heterogeneous Experts: A Scalable MoE Framework with Anisotropy-Preserving Fusion
- 2026Unified Semantic Modeling Framework for Large-Scale Job Understanding at LinkedIn
- 2024HyQE: Ranking Contexts with Hypothetical Query Embeddings
- 2026KARMA: Knowledge-Action Regularized Multimodal Alignment for Personalized Search at Taobao
- 2025OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking System