# Simple Applications of BERT for Ad Hoc Document Retrieval

arXiv: https://arxiv.org/abs/1903.10972 · HTML: https://mlsanity.com/paper/1903.10972/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2019.

## Built on by (most cited first, 30 of 34)

- [LLMs4OL: Large Language Models for Ontology Learning](https://mlsanity.com/paper/2307.16648/) (2023)
- [THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption](https://mlsanity.com/paper/2206.00216/) (2022)
- [Benchmarking Chinese Text Recognition: Datasets, Baselines, and an Empirical Study](https://mlsanity.com/paper/2112.15093/) (2021)
- [Bi-PointFlowNet: Bidirectional Learning for Point Cloud Based Scene Flow Estimation](https://mlsanity.com/paper/2207.07522/) (2022)
- [Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Models](https://mlsanity.com/paper/2209.06506/) (2022)
- [Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models](https://mlsanity.com/paper/2201.08471/) (2022)
- [A Review of Deep Learning for Video Captioning](https://mlsanity.com/paper/2304.11431/) (2023)
- [B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval](https://mlsanity.com/paper/2104.09791/) (2021)
- [On the Calibration and Uncertainty of Neural Learning to Rank Models](https://mlsanity.com/paper/2101.04356/) (2021)
- [Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges](https://mlsanity.com/paper/2403.00784/) (2024)
- [Assisting the Human Fact-Checkers: Detecting All Previously Fact-Checked Claims in a Document](https://mlsanity.com/paper/2109.07410/) (2021)
- [Neural ranking models for document retrieval](https://mlsanity.com/paper/2102.11903/) (2021)
- [Article Reranking by Memory-Enhanced Key Sentence Matching for Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2112.10322/) (2021)
- [Enhancing Q&A Text Retrieval with Ranking Models: Benchmarking, fine-tuning and deploying Rerankers for RAG](https://mlsanity.com/paper/2409.07691/) (2024)
- [MINERS: Multilingual Language Models as Semantic Retrievers](https://mlsanity.com/paper/2406.07424/) (2024)
- [Discovering Explanatory Sentences in Legal Case Decisions Using Pre-trained Language Models](https://mlsanity.com/paper/2112.07165/) (2021)
- [Large Language Models for Relevance Judgment in Product Search](https://mlsanity.com/paper/2406.00247/) (2024)
- [Domain-Specific Pretraining for Vertical Search: Case Study on Biomedical Literature](https://mlsanity.com/paper/2106.13375/) (2021)
- [Glitter or Gold? Deriving Structured Insights from Sustainability Reports via Large Language Models](https://mlsanity.com/paper/2310.05628/) (2023)
- [Synthetic Document Generator for Annotation-free Layout Recognition](https://mlsanity.com/paper/2111.06016/) (2021)
- [Diagnosing BERT with Retrieval Heuristics](https://mlsanity.com/paper/2201.04458/) (2022)
- [Cross-Lingual Training with Dense Retrieval for Document Retrieval](https://mlsanity.com/paper/2109.01628/) (2021)
- [Multistage BiCross encoder for multilingual access to COVID-19 health information](https://mlsanity.com/paper/2101.03013/) (2021)
- [News Article Retrieval in Context for Event-centric Narrative Creation](https://mlsanity.com/paper/2106.16053/) (2021)
- [MetricBERT: Text Representation Learning via Self-Supervised Triplet Training](https://mlsanity.com/paper/2208.06610/) (2022)
- [A Neural Model for Joint Document and Snippet Ranking in Question Answering for Large Document Collections](https://mlsanity.com/paper/2106.08908/) (2021)
- [Enhancing ID-based Recommendation with Large Language Models](https://mlsanity.com/paper/2411.02041/) (2024)
- [An Analysis of a BERT Deep Learning Strategy on a Technology Assisted Review Task](https://mlsanity.com/paper/2104.08340/) (2021)
- [Topics in Contextualised Attention Embeddings](https://mlsanity.com/paper/2301.04339/) (2023)
- [COVID-19: Comparative Analysis of Methods for Identifying Articles Related to Therapeutics and Vaccines without Using Labeled Data](https://mlsanity.com/paper/2101.02017/) (2021)

## Abstract (arXiv)

Following recent successes in applying BERT to question answering, we explore simple applications to ad hoc document retrieval. This required confronting the challenge posed by documents that are typically longer than the length of input BERT was designed to handle. We address this issue by applying inference on sentences individually, and then aggregating sentence scores to produce document scores. Experiments on TREC microblog and newswire test collections show that our approach is simple yet effective, as we report the highest average precision on these datasets by neural approaches that we are aware of.