# Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

arXiv: https://arxiv.org/abs/1908.08167 · HTML: https://mlsanity.com/paper/1908.08167/

## Key facts

- Cited by 96 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2019.

## Built on by (most cited first, 30 of 96)

- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [RealTime QA: What's the Answer Right Now?](https://mlsanity.com/paper/2207.13332/) (2022)
- [Efficient Training of Audio Transformers with Patchout](https://mlsanity.com/paper/2110.05069/) (2021)
- [Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering](https://mlsanity.com/paper/2101.00774/) (2021)
- [RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking](https://mlsanity.com/paper/2110.07367/) (2021)
- [Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering](https://mlsanity.com/paper/2307.16877/) (2023)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering](https://mlsanity.com/paper/2106.05346/) (2021)
- [Transformers are Multi-State RNNs](https://mlsanity.com/paper/2401.06104/) (2024)
- [TopiOCQA: Open-domain Conversational Question Answering with Topic Switching](https://mlsanity.com/paper/2110.00768/) (2021)
- [Learning to Filter Context for Retrieval-Augmented Generation](https://mlsanity.com/paper/2311.08377/) (2023)
- [Efficient Long-Text Understanding with Short-Text Models](https://mlsanity.com/paper/2208.00748/) (2022)
- [KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering](https://mlsanity.com/paper/2110.04330/) (2021)
- [Natural Language Video Localization: A Revisit in Span-based Question Answering Framework](https://mlsanity.com/paper/2102.13558/) (2021)
- [Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?](https://mlsanity.com/paper/2401.11911/) (2024)
- [A Survey on Long Text Modeling with Transformers](https://mlsanity.com/paper/2302.14502/) (2023)
- [Multi-task Retrieval for Knowledge-Intensive Tasks](https://mlsanity.com/paper/2101.00117/) (2021)
- [UnitedQA: A Hybrid Approach for Open Domain Question Answering](https://mlsanity.com/paper/2101.00178/) (2021)
- [VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling](https://mlsanity.com/paper/2406.04321/) (2024)
- [TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection](https://mlsanity.com/paper/2411.02886/) (2024)
- [Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Models](https://mlsanity.com/paper/2503.15888/) (2025)
- [R2-D2: A Modular Baseline for Open-Domain Question Answering](https://mlsanity.com/paper/2109.03502/) (2021)
- [An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2210.16773/) (2022)
- [Improving Medical Reasoning through Retrieval and Self-Reflection with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2401.15269/) (2024)
- [Grounding Language Model with Chunking-Free In-Context Retrieval](https://mlsanity.com/paper/2402.09760/) (2024)
- [Adaptive Information Seeking for Open-Domain Question Answering](https://mlsanity.com/paper/2109.06747/) (2021)
- [Selective Attention Improves Transformer](https://mlsanity.com/paper/2410.02703/) (2024)
- [Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering](https://mlsanity.com/paper/2108.02866/) (2021)
- [OLAPH: Improving Factuality in Biomedical Long-form Question Answering](https://mlsanity.com/paper/2405.12701/) (2024)
- [Phaseformer: Phase-based Attention Mechanism for Underwater Image Restoration and Beyond](https://mlsanity.com/paper/2412.01456/) (2024)

## Abstract (arXiv)

BERT model has been successfully applied to open-domain QA tasks. However, previous work trains BERT by viewing passages corresponding to the same question as independent training instances, which may cause incomparable scores for answers from different passages. To tackle this issue, we propose a multi-passage BERT model to globally normalize answer scores across all passages of the same question, and this change enables our QA model find better answers by utilizing more passages. In addition, we find that splitting articles into passages with the length of 100 words by sliding window improves performance by 4%. By leveraging a passage ranker to select high-quality passages, multi-passage BERT gains additional 2%. Experiments on four standard benchmarks showed that our multi-passage BERT outperforms all state-of-the-art models on all benchmarks. In particular, on the OpenSQuAD dataset, our model gains 21.4% EM and 21.5% F1 over all non-BERT models, and 5.8% EM and 6.5% F1 over BERT-based models.