# DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding

arXiv: https://arxiv.org/abs/2002.12591 · HTML: https://mlsanity.com/paper/2002.12591/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2020.

## Built on by (most cited first, 20 of 20)

- [Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering](https://mlsanity.com/paper/2101.00774/) (2021)
- [How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2203.16262/) (2022)
- [Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCo](https://mlsanity.com/paper/2203.17248/) (2022)
- [Decoupled Adversarial Contrastive Learning for Self-supervised Adversarial Robustness](https://mlsanity.com/paper/2207.10899/) (2022)
- [Exploring Dense Retrieval for Dialogue Response Selection](https://mlsanity.com/paper/2110.06612/) (2021)
- [EmbedDistill: A Geometric Knowledge Distillation for Information Retrieval](https://mlsanity.com/paper/2301.12005/) (2023)
- [On the Pros and Cons of Momentum Encoder in Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/2208.05744/) (2022)
- [SDR: Efficient Neural Re-ranking using Succinct Document Representation](https://mlsanity.com/paper/2110.02065/) (2021)
- [Generalized Contrastive Learning for Multi-Modal Retrieval and Ranking](https://mlsanity.com/paper/2404.08535/) (2024)
- [Prior-Fitted Networks Scale to Larger Datasets When Treated as Weak Learners](https://mlsanity.com/paper/2503.01256/) (2025)
- [DimCL: Dimensional Contrastive Learning For Improving Self-Supervised Learning](https://mlsanity.com/paper/2309.11782/) (2023)
- [Contextualizing Meta-Learning via Learning to Decompose](https://mlsanity.com/paper/2106.08112/) (2021)
- [Once is Enough: A Light-Weight Cross-Attention for Fast Sentence Pair Modeling](https://mlsanity.com/paper/2210.05261/) (2022)
- [Bootstrapped Pre-training with Dynamic Identifier Prediction for Generative Retrieval](https://mlsanity.com/paper/2407.11504/) (2024)
- [Multimodal semantic retrieval for product search](https://mlsanity.com/paper/2501.07365/) (2025)
- [Decoupled Transformer for Scalable Inference in Open-domain Question Answering](https://mlsanity.com/paper/2108.02765/) (2021)
- [Geo-Encoder: A Chunk-Argument Bi-Encoder Framework for Chinese Geographic Re-Ranking](https://mlsanity.com/paper/2309.01606/) (2023)
- [Automating AI Failure Tracking: Semantic Association of Reports in AI Incident Database](https://mlsanity.com/paper/2507.23669/) (2025)
- [Roof-Transformer: Divided and Joined Understanding with Knowledge Enhancement](https://mlsanity.com/paper/2112.06736/) (2021)
- [Brand Celebrity Matching Model Based on Natural Language Processing](https://mlsanity.com/paper/2208.08887/) (2022)

## Abstract (arXiv)

Recent studies on open-domain question answering have achieved prominent performance improvement using pre-trained language models such as BERT. State-of-the-art approaches typically follow the "retrieve and read" pipeline and employ BERT-based reranker to filter retrieved documents before feeding them into the reader module. The BERT retriever takes as input the concatenation of question and each retrieved document. Despite the success of these approaches in terms of QA accuracy, due to the concatenation, they can barely handle high-throughput of incoming questions each with a large collection of retrieved documents. To address the efficiency problem, we propose DC-BERT, a decoupled contextual encoding framework that has dual BERT models: an online BERT which encodes the question only once, and an offline BERT which pre-encodes all the documents and caches their encodings. On SQuAD Open and Natural Questions Open datasets, DC-BERT achieves 10x speedup on document retrieval, while retaining most (about 98%) of the QA performance compared to state-of-the-art approaches for open-domain question answering.