# Quality-Weighted Vendi Scores And Their Application To Diverse Experimental Design

arXiv: https://arxiv.org/abs/2405.02449 · HTML: https://mlsanity.com/paper/2405.02449/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 3 May 2024.

## Builds on (most cited first, 13 of 13)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning](https://mlsanity.com/paper/2311.08182/) (2023)
- [Discovering Many Diverse Solutions with Bayesian Optimization](https://mlsanity.com/paper/2210.10953/) (2022)
- [Active model learning and diverse action sampling for task and motion planning](https://mlsanity.com/paper/1803.00967/) (2018)
- [Diversified Sampling for Batched Bayesian Optimization with Determinantal Point Processes](https://mlsanity.com/paper/2110.11665/) (2021)
- [Cousins Of The Vendi Score: A Family Of Similarity-Based Diversity Metrics For Science And Machine Learning](https://mlsanity.com/paper/2310.12952/) (2023)
- [Discovering Valuable Items from Massive Data](https://mlsanity.com/paper/1506.00935/) (2015)
- [Towards Mode Balancing of Generative Models via Diversity Weights](https://mlsanity.com/paper/2304.11961/) (2023)
- [Nonmyopic Multiclass Active Search with Diminishing Returns for Diverse Discovery](https://mlsanity.com/paper/2202.03593/) (2022)

## Built on by (most cited first, 17 of 17)

- [Beyond Aesthetics: Cultural Competence in Text-to-Image Models](https://mlsanity.com/paper/2407.06863/) (2024)
- [Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models](https://mlsanity.com/paper/2408.02085/) (2024)
- [Diffusion Models Through a Global Lens: Are They Culturally Inclusive?](https://mlsanity.com/paper/2502.08914/) (2025)
- [Vendi-RAG: Adaptively Trading-Off Diversity And Quality Significantly Improves Retrieval Augmented Generation With LLMs](https://mlsanity.com/paper/2502.11228/) (2025)
- [The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance](https://mlsanity.com/paper/2502.04593/) (2025)
- [The Vendiscope: An Algorithmic Microscope For Data Collections](https://mlsanity.com/paper/2502.10828/) (2025)
- [AutoQD: Automatic Discovery of Diverse Behaviors with Quality-Diversity Optimization](https://mlsanity.com/paper/2506.05634/) (2025)
- [Amortized nonmyopic active search via deep imitation learning](https://mlsanity.com/paper/2405.15031/) (2024)
- [Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error](https://mlsanity.com/paper/2602.14682/) (2026)
- [Soft Quality-Diversity Optimization](https://mlsanity.com/paper/2512.00810/) (2025)
- [Vendi Novelty Scores for Out-of-Distribution Detection](https://mlsanity.com/paper/2602.10062/) (2026)
- [Mixture-Greedy for Online Generative Model Selection: Is UCB Necessary in Diversity-Aware Multi-Armed Bandits?](https://mlsanity.com/paper/2603.21716/) (2026)
- [Vendi Information Gain for Active Learning and its Application to Ecology](https://mlsanity.com/paper/2509.10390/) (2025)
- [Quality-Diversity Optimization as Multi-Objective Optimization](https://mlsanity.com/paper/2602.00478/) (2026)
- [Building informative materials datasets beyond targeted objectives](https://mlsanity.com/paper/2605.05104/) (2026)
- [Spokes: Optimizing for Diverse Pretraining Data Selection](https://mlsanity.com/paper/2606.15216/) (2026)
- [Support or Magnitude? Characterizing Neuron Relevance for Instruction Data Selection](https://mlsanity.com/paper/2610.16531/) (2026)

## Abstract (arXiv)

Experimental design techniques such as active search and Bayesian optimization are widely used in the natural sciences for data collection and discovery. However, existing techniques tend to favor exploitation over exploration of the search space, which causes them to get stuck in local optima. This ``collapse" problem prevents experimental design algorithms from yielding diverse high-quality data. In this paper, we extend the Vendi scores -- a family of interpretable similarity-based diversity metrics -- to account for quality. We then leverage these quality-weighted Vendi scores to tackle experimental design problems across various applications, including drug discovery, materials discovery, and reinforcement learning. We found that quality-weighted Vendi scores allow us to construct policies for experimental design that flexibly balance quality and diversity, and ultimately assemble rich and diverse sets of high-performing data points. Our algorithms led to a 70%-170% increase in the number of effective discoveries compared to baselines.