# SEED-SET: Scalable Evolving Experimental Design for System-level Ethical Testing

arXiv: https://arxiv.org/abs/2603.01630 · HTML: https://mlsanity.com/paper/2603.01630/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 64 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Builds on (most cited first, 30 of 30)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Self-Rewarding Language Models](https://mlsanity.com/paper/2401.10020/) (2024)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators](https://mlsanity.com/paper/2403.16950/) (2024)
- [Can Machines Learn Morality? The Delphi Experiment](https://mlsanity.com/paper/2110.07574/) (2021)
- [The Ethics of AI Ethics -- An Evaluation of Guidelines](https://mlsanity.com/paper/1903.03425/) (2019)
- [Modern Bayesian Experimental Design](https://mlsanity.com/paper/2302.14545/) (2023)
- [AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies](https://mlsanity.com/paper/2407.17436/) (2024)
- [Artificial Intelligence Index Report 2025](https://mlsanity.com/paper/2504.07139/) (2025)
- [Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge](https://mlsanity.com/paper/2502.00561/) (2025)
- [Measurement to Meaning: A Validity-Centered Framework for AI Evaluation](https://mlsanity.com/paper/2505.10573/) (2025)
- [Open Problems in Technical AI Governance](https://mlsanity.com/paper/2407.14981/) (2024)
- [Bayesian Optimization of Composite Functions](https://mlsanity.com/paper/1906.01537/) (2019)
- [Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes](https://mlsanity.com/paper/2203.11382/) (2022)
- [The Variational Gaussian Process](https://mlsanity.com/paper/1511.06499/) (2015)
- [Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation](https://mlsanity.com/paper/2406.11852/) (2024)
- [The Dark Side of Dataset Scaling: Evaluating Racial Classification in Multimodal Models](https://mlsanity.com/paper/2405.04623/) (2024)
- [Large-scale moral machine experiment on large language models](https://mlsanity.com/paper/2411.06790/) (2024)
- [Accelerated Preference Elicitation with LLM-Based Proxies](https://mlsanity.com/paper/2501.14625/) (2025)
- [Eliciting User Preferences for Personalized Multi-Objective Decision Making through Comparative Feedback](https://mlsanity.com/paper/2302.03805/) (2023)
- [On the Pros and Cons of Active Learning for Moral Preference Elicitation](https://mlsanity.com/paper/2407.18889/) (2024)
- [Towards the Quantification of Safety Risks in Deep Neural Networks](https://mlsanity.com/paper/2009.06114/) (2020)
- [Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Ways](https://mlsanity.com/paper/2402.04420/) (2024)
- [Risk-aware Classification via Uncertainty Quantification](https://mlsanity.com/paper/2412.03391/) (2024)
- [A toolkit of dilemmas: Beyond debiasing and fairness formulas for responsible AI/ML](https://mlsanity.com/paper/2303.01930/) (2023)
- [Assessing Trustworthiness of Autonomous Systems](https://mlsanity.com/paper/2305.03411/) (2023)

## Built on by (most cited first, 1 of 1)

- [Coverage Aware Active Evaluation for Failure Discovery with Paired Systems](https://mlsanity.com/paper/2608.13719/) (2026)

## Abstract (arXiv)

As autonomous systems such as drones, become increasingly deployed in high-stakes, human-centric domains, it is critical to evaluate the ethical alignment since failure to do so imposes imminent danger to human lives, and long term bias in decision-making. Automated ethical benchmarking of these systems is understudied due to the lack of ubiquitous, well-defined metrics for evaluation, and stakeholder-specific subjectivity, which cannot be modeled analytically. To address these challenges, we propose SEED-SET, a Bayesian experimental design framework that incorporates domain-specific objective evaluations, and subjective value judgments from stakeholders. SEED-SET models both evaluation types separately with hierarchical Gaussian Processes, and uses a novel acquisition strategy to propose interesting test candidates based on learnt qualitative preferences and objectives that align with the stakeholder preferences. We validate our approach for ethical benchmarking of autonomous agents on two applications and find our method to perform the best. Our method provides an interpretable and efficient trade-off between exploration and exploitation, by generating up to 2× optimal test candidates compared to baselines, with 1.25× improvement in coverage of high dimensional search spaces.