mlsanityData to 7 Oct 2026

Pre-trained Language Model for Web-scale Retrieval in Baidu Search

arXiv 2106.03373 ↗First posted 7 Jun 2021cs.IRcited by 31

Not on the map. Why?

Abstract (arXiv)

Retrieval is a crucial stage in web search that identifies a small set of query-relevant candidates from a billion-scale corpus. Discovering more semantically-related candidates in the retrieval stage is very promising to expose more high-quality results to the end users. However, it still remains non-trivial challenges of building and deploying effective retrieval models for semantic matching in real search engine. In this paper, we describe the retrieval system that we developed and deployed in Baidu Search. The system exploits the recent state-of-the-art Chinese pretrained language model, namely Enhanced Representation through kNowledge IntEgration (ERNIE), which facilitates the system with expressive semantic matching. In particular, we developed an ERNIE-based retrieval model, which is equipped with 1) expressive Transformer-based semantic encoders, and 2) a comprehensive multi-stage training paradigm. More importantly, we present a practical system workflow for deploying the model in web-scale retrieval. Eventually, the system is fully deployed into production, where rigorous offline and online experiments were conducted. The results show that the system can perform high-quality candidate retrieval, especially for those tail queries with uncommon demands. Overall, the new retrieval system facilitated by pretrained language model (i.e., ERNIE) can largely improve the usability and applicability of our search engine.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jun 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 9; 2023: 10; 2024: 5; 2025: 3; 2026: 3.

31 papers.

Sort:

By area:

  1. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 141
  2. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 89
  3. 2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
  4. 2023Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Trainingcited by 35
  5. 2022HLATR: Enhance Multi-stage Text Retrieval with Hybrid List Aware Transformer Rerankingcited by 19
  6. 2024Enhancing Recommendation Diversity by Re-ranking with Large Language Modelscited by 19
  7. 2022Uni-Retriever: Towards Learning The Unified Embedding Based Retriever in Bing Sponsored Searchcited by 17
  8. 2022Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddingscited by 17
  9. 2025TongSearch-QR: Reinforced Query Reasoning for Retrievalcited by 11
  10. 2022Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-rankingcited by 10
  11. 2024Multimodal Query Suggestion with Multi-Agent Reinforcement Learning from Human Feedbackcited by 9
  12. 2022Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrievalcited by 7
19 more
  1. 2023Pre-training with Large Language Model-based Document Expansion for Dense Passage Retrievalcited by 7
  2. 2023ESPN: Memory-Efficient Multi-Vector Information Retrievalcited by 7
  3. 2025LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Servicescited by 7
  4. 2022Modality-Balanced Embedding for Video Retrievalcited by 6
  5. 2024NUDGE: Lightweight Non-Parametric Fine-Tuning of Embeddings for Retrievalcited by 5
  6. 2022PENTATRON: PErsonalized coNText-Aware Transformer for Retrieval-based cOnversational uNderstandingcited by 3
  7. 2024Centrality-aware Product Retrieval and Rankingcited by 3
  8. 2024Best Practices for Distilling Large Language Models into BERT for Web Search Rankingcited by 3
  9. 2025CoVE: Compressed Vocabulary Expansion Makes Better LLM-based Recommender Systemscited by 3
  10. 2023Integrity and Junkiness Failure Handling for Embedding-based Retrieval: A Case Study in Social Network Searchcited by 2
  11. 2023LEA: Improving Sentence Similarity Robustness to Typos Using Lexical Attention Biascited by 2
  12. 2021Intent-based Product Collections for E-commerce using Pretrained Language Modelscited by 1
  13. 2022NeuralSearchX: Serving a Multi-billion-parameter Reranker for Multilingual Metasearch at a Low Costcited by 1
  14. 2023Event-driven Real-time Retrieval in Web Searchcited by 1
  15. 2023Layout-aware Webpage Quality Assessmentcited by 0
  16. 2023Generate to Understand for Representationcited by 0
  17. 2026Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillationcited by 0
  18. 2026PLUME: Parameter-Efficient Personalization of Large Language Models via Low-Rank User Modulation in Shared Subspacescited by 0
  19. 2026Gradient-Aligned Pair Selection for Personalized Preference Optimizationcited by 0