mlsanityData to 7 Oct 2026

Deeper Text Understanding for IR with Contextual Neural Language Modeling

arXiv 1905.09217 ↗First posted 22 May 2019cs.IR cs.CLcited by 111

Not on the map. Why?

Abstract (arXiv)

Neural networks provide new possibilities to automatically learn complex language patterns and query-document relations. Neural IR models have achieved promising results in learning query-document relevance patterns, but few explorations have been done on understanding the text content of a query or a document. This paper studies leveraging a recently-proposed contextual neural language model, BERT, to provide deeper text understanding for IR. Experimental results demonstrate that the contextual text representations from BERT are more effective than traditional word embeddings. Compared to bag-of-words retrieval models, the contextual language model can better leverage language structures, bringing large improvements on queries written in natural languages. Combining the text understanding ability with search knowledge leads to an enhanced pre-trained BERT model that can benefit related search tasks where training data are limited.

Key facts. Cited by 111 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 31; 2023: 13; 2024: 19; 2025: 15; 2026: 10.

111 papers.

Sort:

By area:

  1. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  2. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  3. 2022Task-aware Retrieval with Instructionscited by 107
  4. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
  5. 2021Adversarial Retriever-Ranker for dense text retrievalcited by 82
  6. 2021The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Modelscited by 81
  7. 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 67
  8. 2022InPars: Data Augmentation for Information Retrieval using Large Language Modelscited by 60
  9. 2021PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrievalcited by 50
  10. 2022Retrieval-Enhanced Machine Learningcited by 46
  11. 2022Multi-View Document Representation Learning for Open-Domain Dense Retrievalcited by 41
  12. 2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
38 more
  1. 2022Towards Robust Ranker for Text Retrievalcited by 37
  2. 2022Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Modelscited by 37
  3. 2023QUEST: A Retrieval Dataset of Entity-Seeking Queries with Implicit Set Operationscited by 30
  4. 2023Black-box Adversarial Attacks against Dense Retrieval Models: A Multi-view Contrastive Learning Methodcited by 29
  5. 2024PipeRAG: Fast Retrieval-Augmented Generation via Algorithm-System Co-designcited by 28
  6. 2022Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrievalcited by 26
  7. 2021B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrievalcited by 25
  8. 2021On Cross-Lingual Retrieval with Multilingual Text Encoderscited by 22
  9. 2024Translate-Distill: Learning Cross-Language Dense Retrieval by Translation and Distillationcited by 22
  10. 2022SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrievalcited by 21
  11. 2022PROD: Progressive Distillation for Dense Retrievalcited by 20
  12. 2024Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challengescited by 20
  13. 2021Corpus-Level Evaluation for Event QA: The IndiaPoliceEvents Corpus Covering the 2002 Gujarat Violencecited by 19
  14. 2022Neural Approaches to Multilingual Information Retrievalcited by 18
  15. 2023Agent4Ranking: Semantic Robust Ranking via Personalized Query Rewriting Using Multi-agent LLMcited by 18
  16. 2021Neural ranking models for document retrievalcited by 16
  17. 2025Hierarchical Document Refinement for Long-context Retrieval-augmented Generationcited by 16
  18. 2022DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Enginecited by 15
  19. 2023Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboardcited by 15
  20. 2024RankMamba: Benchmarking Mamba's Document Ranking Performance in the Era of Transformerscited by 15
  21. 2024Learning Interpretable Legal Case Retrieval via Knowledge-Guided Case Reformulationcited by 15
  22. 2022C3: Continued Pretraining with Contrastive Weak Supervision for Cross Language Ad-Hoc Retrievalcited by 14
  23. 2022Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Surveycited by 14
  24. 2023Context Aware Query Rewriting for Text Rankers using LLMcited by 14
  25. 2024Multi-granular Adversarial Attacks against Black-box Neural Ranking Modelscited by 14
  26. 2024Dwell in the Beginning: How Language Models Embed Long Documents for Dense Retrievalcited by 13
  27. 2025AssertionForge: Enhancing Formal Verification Assertion Generation with Structured Representation of Specifications and RTLcited by 13
  28. 2023MGeo: Multi-Modal Geographic Pre-Training Methodcited by 12
  29. 2023EmbedDistill: A Geometric Knowledge Distillation for Information Retrievalcited by 11
  30. 2025Time Series Language Model for Descriptive Caption Generationcited by 11
  31. 2021Intra-Document Cascading: Learning to Select Passages for Neural Document Rankingcited by 10
  32. 2022DPTDR: Deep Prompt Tuning for Dense Passage Retrievalcited by 10
  33. 2024State Space Models are Strong Text Rerankerscited by 10
  34. 2025CSPLADE: Learned Sparse Retrieval with Causal Language Modelscited by 10
  35. 2022Long Document Re-ranking with Modular Re-rankercited by 9
  36. 2022P^3 Ranker: Mitigating the Gaps between Pre-training and Ranking Fine-tuning with Prompt-based Learning and Pre-finetuningcited by 7
  37. 2022MASTER: Multi-task Pre-trained Bottlenecked Masked Autoencoders are Better Dense Retrieverscited by 7
  38. 2023Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Offcited by 7

50 most cited of 111 All 111 (JSON)