# Collaborative Intelligence in Sequential Experiments: A Human-in-the-Loop Framework for Drug Discovery

arXiv: https://arxiv.org/abs/2405.03942 · HTML: https://mlsanity.com/paper/2405.03942/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 72 references that are in the map.
- First posted on arXiv 7 May 2024.

## Builds on (most cited first, 16 of 16)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Learning to Complement Humans](https://mlsanity.com/paper/2005.00582/) (2020)
- [Differentiable Learning Under Triage](https://mlsanity.com/paper/2103.08902/) (2021)
- [Improving Expert Predictions with Conformal Prediction](https://mlsanity.com/paper/2201.12006/) (2022)
- [Human-Algorithm Collaboration: Achieving Complementarity and Avoiding Unfairness](https://mlsanity.com/paper/2202.08821/) (2022)
- [Classification Under Human Assistance](https://mlsanity.com/paper/2006.11845/) (2020)
- [Human-AI Collaboration with Bandit Feedback](https://mlsanity.com/paper/2105.10614/) (2021)
- [Review on Ranking and Selection: A New Perspective](https://mlsanity.com/paper/2008.00249/) (2020)
- [The Assistive Multi-Armed Bandit](https://mlsanity.com/paper/1901.08654/) (2019)
- [Context-Based Dynamic Pricing with Online Clustering](https://mlsanity.com/paper/1902.06199/) (2019)
- [A Primal-dual Learning Algorithm for Personalized Dynamic Pricing with an Inventory Constraint](https://mlsanity.com/paper/1812.09234/) (2018)
- [Speed Up the Cold-Start Learning in Two-Sided Bandits with Many Arms](https://mlsanity.com/paper/2210.00340/) (2022)

## Built on by (most cited first, 3 of 3)

- [Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models](https://mlsanity.com/paper/2501.05382/) (2025)
- [GRASP: Graph Reasoning Agents for Systems Pharmacology with Human-in-the-Loop](https://mlsanity.com/paper/2512.05502/) (2025)
- [LLM-Augmented Digital Twin for Policy Evaluation in Short-Video Platforms](https://mlsanity.com/paper/2603.11333/) (2026)

## Abstract (arXiv)

Drug discovery is a complex process that involves sequentially screening and examining a vast array of molecules to identify those with the target properties. This process, also referred to as sequential experimentation, faces challenges due to the vast search space, the rarity of target molecules, and constraints imposed by limited data and experimental budgets. To address these challenges, we introduce a human-in-the-loop framework for sequential experiments in drug discovery. This collaborative approach combines human expert knowledge with deep learning algorithms, enhancing the discovery of target molecules within a specified experimental budget. The proposed algorithm processes experimental data to recommend both promising molecules and those that could improve its performance to human experts. Human experts retain the final decision-making authority based on these recommendations and their domain expertise, including the ability to override algorithmic recommendations. We applied our method to drug discovery tasks using real-world data and found that it consistently outperforms all baseline methods, including those which rely solely on human or algorithmic input. This demonstrates the complementarity between human experts and the algorithm. Our results provide key insights into the levels of humans' domain knowledge, the importance of meta-knowledge, and effective work delegation strategies. Our findings suggest that such a framework can significantly accelerate the development of new vaccines and drugs by leveraging the best of both human and artificial intelligence.