mlsanityData to 7 Oct 2026

Understanding the Behaviors of BERT in Ranking

arXiv 1904.07531 ↗First posted 16 Apr 2019cs.IR cs.CLcited by 49

Not on the map. Why?

Abstract (arXiv)

This paper studies the performances and behaviors of BERT in ranking tasks. We explore several different ways to leverage the pre-trained BERT and fine-tune it on two ranking tasks: MS MARCO passage reranking and TREC Web Track ad hoc document ranking. Experimental results on MS MARCO demonstrate the strong effectiveness of BERT in question-answering focused passage ranking tasks, as well as the fact that BERT is a strong interaction-based seq2seq matching model. Experimental results on TREC show the gaps between the BERT pre-trained on surrounding contexts and the needs of ad hoc document ranking. Analyses illustrate how BERT allocates its attentions between query-document tokens in its Transformer layers, how it prefers semantic matches between paraphrase tokens, and how that differs with the soft match patterns learned by a click-trained neural ranker.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 15; 2022: 11; 2023: 10; 2024: 8; 2025: 5.

49 papers.

Sort:

By area:

  1. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  2. 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 91
  3. 2021HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrievalcited by 87
  4. 2022Curriculum Learning for Dense Retrieval Distillationcited by 38
  5. 2021R2-D2: A Modular Baseline for Open-Domain Question Answeringcited by 37
  6. 2022Towards Robust Ranker for Text Retrievalcited by 37
  7. 2023A Preliminary Study of the Intrinsic Relationship between Complexity and Alignmentcited by 33
  8. 2024A Semantic Search Engine for Mathlib4cited by 30
  9. 2025Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Surveycited by 29
  10. 2021Rider: Reader-Guided Passage Reranking for Open-Domain Question Answeringcited by 28
  11. 2023Learning to Relate to Previous Turns in Conversational Searchcited by 26
  12. 2022Emotion Based Hate Speech Detection using Multimodal Learningcited by 18
37 more
  1. 2021Neural ranking models for document retrievalcited by 16
  2. 2024Enhancing Q&A Text Retrieval with Ranking Models: Benchmarking, fine-tuning and deploying Rerankers for RAGcited by 16
  3. 2021Natural Language Processing for Smart Healthcarecited by 14
  4. 2023GripRank: Bridging the Gap between Retrieval and Generation via the Generative Knowledge Improved Passage Rankingcited by 11
  5. 2021Improving Pretrained Models for Zero-shot Multi-label Text Classification through Reinforced Label Hierarchy Reasoningcited by 9
  6. 2021Mixed Attention Transformer for Leveraging Word-Level Knowledge to Neural Cross-Lingual Information Retrievalcited by 8
  7. 2022P^3 Ranker: Mitigating the Gaps between Pre-training and Ranking Fine-tuning with Prompt-based Learning and Pre-finetuningcited by 7
  8. 2021Pruning the Index Contents for Memory Efficient Open-Domain QAcited by 6
  9. 2022Paying More Attention to Self-attention: Improving Pre-trained Language Models via Attention Guidingcited by 6
  10. 2023Generating Natural Language Queries for More Effective Systematic Review Screening Prioritisationcited by 6
  11. 2025Pathway to Relevance: How Cross-Encoders Implement a Semantic Variant of BM25cited by 6
  12. 2021Training Adaptive Computation for Open-Domain Question Answering with Computational Constraintscited by 4
  13. 2024Hard Cases Detection in Motion Prediction by Vision-Language Foundation Modelscited by 4
  14. 2024Deep Bag-of-Words Model: An Efficient and Interpretable Relevance Architecture for Chinese E-Commercecited by 4
  15. 2025Learning an Effective Premise Retrieval Model for Efficient Mathematical Formalizationcited by 4
  16. 2021News Article Retrieval in Context for Event-centric Narrative Creationcited by 3
  17. 2021Modeling Relevance Ranking under the Pre-training and Fine-tuning Paradigmcited by 2
  18. 2022Block-SCL: Blocking Matters for Supervised Contrastive Learning in Product Matchingcited by 2
  19. 2022Pre-training for Information Retrieval: Are Hyperlinks Fully Explored?cited by 2
  20. 2023GPT-4 and Safety Case Generation: An Exploratory Analysiscited by 2
  21. 2022Retrieval Augmentation for T5 Re-ranker using External Sourcescited by 1
  22. 2023Characterizing and Classifying Developer Forum Posts with their Intentionscited by 1
  23. 2024Text Categorization Can Enhance Domain-Agnostic Stopword Extractioncited by 1
  24. 2024HNCSE: Advancing Sentence Embeddings via Hybrid Contrastive Learning with Hard Negativescited by 1
  25. 2021A multi-perspective combined recall and rank framework for Chinese procedure terminology normalizationcited by 0
  26. 2021BERT Embeddings Can Track Context in Conversational Searchcited by 0
  27. 2021TPRM: A Topic-based Personalized Ranking Model for Web Searchcited by 0
  28. 2022How Different are Pre-trained Transformers for Text Ranking?cited by 0
  29. 2022Sentiment Classification of Code-Switched Text using Pre-trained Multilingual Embeddings and Segmentationcited by 0
  30. 2022Explainability of Text Processing and Retrieval Methods: A Surveycited by 0
  31. 2023Improving Text Matching in E-Commerce Search with A Rationalizable, Intervenable and Fast Entity-Based Relevance Modelcited by 0
  32. 2023A Deep Reinforcement Learning Approach for Interactive Search with Sentence-level Feedbackcited by 0
  33. 2023Latent Feature-based Data Splits to Improve Generalisation Evaluation: A Hate Speech Detection Case Studycited by 0
  34. 2024Domain Adaptation of Multilingual Semantic Search -- Literature Reviewcited by 0
  35. 2024E2E-AFG: An End-to-End Model with Adaptive Filtering for Retrieval-Augmented Generationcited by 0
  36. 2025PRECTR: A Synergistic Framework for Integrating Personalized Search Relevance Matching and CTR Predictioncited by 0
  37. 2025Personalized Product Search Ranking: A Multi-Task Learning Approach with Tabular and Non-Tabular Datacited by 0