# Autonomy and Reliability of Continuous Active Learning for Technology-Assisted Review

arXiv: https://arxiv.org/abs/1504.06868 · HTML: https://mlsanity.com/paper/1504.06868/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2015.

## Built on by (most cited first, 13 of 13)

- [Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?](https://mlsanity.com/paper/2306.13906/) (2023)
- [Overview of the TREC 2022 deep learning track](https://mlsanity.com/paper/2507.10865/) (2025)
- [An Information Retrieval Approach to Building Datasets for Hate Speech Detection](https://mlsanity.com/paper/2106.09775/) (2021)
- [Sentence Embeddings and High-speed Similarity Search for Fast Computer Assisted Annotation of Legal Documents](https://mlsanity.com/paper/2112.11494/) (2021)
- [Cross-Domain Generalization and Knowledge Transfer in Transformers Trained on Legal Data](https://mlsanity.com/paper/2112.07870/) (2021)
- [Heuristic Stopping Rules For Technology-Assisted Review](https://mlsanity.com/paper/2106.09871/) (2021)
- [Impact of Stop Sets on Stopping Active Learning for Text Classification](https://mlsanity.com/paper/2201.05460/) (2022)
- [Toward an Intelligent Tutoring System for Argument Mining in Legal Texts](https://mlsanity.com/paper/2210.13635/) (2022)
- [Combining Counting Processes and Classification Improves a Stopping Rule for Technology Assisted Review](https://mlsanity.com/paper/2312.03171/) (2023)
- [Finite-Sample Coverage Audits for High-Recall Candidate Generation: Certification and Learning-Theoretic Design](https://mlsanity.com/paper/2607.21480/) (2026)
- [Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?](https://mlsanity.com/paper/2303.09325/) (2023)
- [Opening the TAR Black Box: Developing an Interpretable System for eDiscovery Using the Fuzzy ARTMAP Neural Network](https://mlsanity.com/paper/2305.04237/) (2023)
- [ReLeVAnT: Relevance Lexical Vectors for Accurate Legal Text Classification](https://mlsanity.com/paper/2604.22292/) (2026)

## Abstract (arXiv)

We enhance the autonomy of the continuous active learning method shown by Cormack and Grossman (SIGIR 2014) to be effective for technology-assisted review, in which documents from a collection are retrieved and reviewed, using relevance feedback, until substantially all of the relevant documents have been reviewed. Autonomy is enhanced through the elimination of topic-specific and dataset-specific tuning parameters, so that the sole input required by the user is, at the outset, a short query, topic description, or single relevant document; and, throughout the review, ongoing relevance assessments of the retrieved documents. We show that our enhancements consistently yield superior results to Cormack and Grossman's version of continuous active learning, and other methods, not only on average, but on the vast majority of topics from four separate sets of tasks: the legal datasets examined by Cormack and Grossman, the Reuters RCV1-v2 subject categories, the TREC 6 AdHoc task, and the construction of the TREC 2002 filtering test collection.