# Technology Assisted Reviews: Finding the Last Few Relevant Documents by Asking Yes/No Questions to Reviewers

arXiv: https://arxiv.org/abs/1810.05414 · HTML: https://mlsanity.com/paper/1810.05414/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2018.

## Built on by (most cited first, 4 of 4)

- [Zero-shot Generative Large Language Models for Systematic Review Screening Automation](https://mlsanity.com/paper/2401.06320/) (2024)
- [Category Aware Explainable Conversational Recommendation](https://mlsanity.com/paper/2103.08733/) (2021)
- [Leveraging Knowledge Graph Embedding for Effective Conversational Recommendation](https://mlsanity.com/paper/2408.01342/) (2024)
- [Beyond Static: Related Questions Retrieval Through Conversations in Community Question Answering](https://mlsanity.com/paper/2604.22759/) (2026)

## Abstract (arXiv)

The goal of a technology-assisted review is to achieve high recall with low human effort. Continuous active learning algorithms have demonstrated good performance in locating the majority of relevant documents in a collection, however their performance is reaching a plateau when 80%-90% of them has been found. Finding the last few relevant documents typically requires exhaustively reviewing the collection. In this paper, we propose a novel method to identify these last few, but significant, documents efficiently. Our method makes the hypothesis that entities carry vital information in documents, and that reviewers can answer questions about the presence or absence of an entity in the missing relevance documents. Based on this we devise a sequential Bayesian search method that selects the optimal sequence of questions to ask. The experimental results show that our proposed method can greatly improve performance requiring less reviewing effort.