# Dealing with Expert Bias in Collective Decision-Making

arXiv: https://arxiv.org/abs/2106.13539 · HTML: https://mlsanity.com/paper/2106.13539/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 69 references that are in the map.
- First posted on arXiv 25 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits](https://mlsanity.com/paper/1402.0555/) (2014)
- [Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oracles](https://mlsanity.com/paper/2002.04926/) (2020)
- [Finite-Time Analysis of Kernelised Contextual Bandits](https://mlsanity.com/paper/1309.6869/) (2013)
- [Explore no more: Improved high-probability regret bounds for non-stochastic bandits](https://mlsanity.com/paper/1506.03271/) (2015)
- [Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizability](https://mlsanity.com/paper/2003.12699/) (2020)
- [Practical Contextual Bandits with Regression Oracles](https://mlsanity.com/paper/1803.01088/) (2018)
- [A Survey on Practical Applications of Multi-Armed and Contextual Bandits](https://mlsanity.com/paper/1904.10040/) (2019)
- [Misspecified Linear Bandits](https://mlsanity.com/paper/1704.06880/) (2017)
- [A Survey on Contextual Multi-armed Bandits](https://mlsanity.com/paper/1508.03326/) (2015)
- [Multi-Armed Bandits for Intelligent Tutoring Systems](https://mlsanity.com/paper/1310.3174/) (2013)
- [Adapting to Misspecification in Contextual Bandits with Offline Regression Oracles](https://mlsanity.com/paper/2102.13240/) (2021)
- [Online Gradient Boosting](https://mlsanity.com/paper/1506.04820/) (2015)
- [Boosting for Online Convex Optimization](https://mlsanity.com/paper/2102.09305/) (2021)

## Built on by (most cited first, 4 of 4)

- [ACROCPoLis: A Descriptive Framework for Making Sense of Fairness](https://mlsanity.com/paper/2304.11217/) (2023)
- [Expertise Trees Resolve Knowledge Limitations in Collective Decision-Making](https://mlsanity.com/paper/2305.01063/) (2023)
- [Mitigating Biases in Collective Decision-Making: Enhancing Performance in the Face of Fake News](https://mlsanity.com/paper/2403.08829/) (2024)
- [Optimizing Resources for On-the-Fly Label Estimation with Multiple Unknown Medical Experts](https://mlsanity.com/paper/2510.03954/) (2025)

## Abstract (arXiv)

Quite some real-world problems can be formulated as decision-making problems wherein one must repeatedly make an appropriate choice from a set of alternatives. Multiple expert judgements, whether human or artificial, can help in taking correct decisions, especially when exploration of alternative solutions is costly. As expert opinions might deviate, the problem of finding the right alternative can be approached as a collective decision making problem (CDM) via aggregation of independent judgements. Current state-of-the-art approaches focus on efficiently finding the optimal expert, and thus perform poorly if all experts are not qualified or if they are overly biased, thereby potentially derailing the decision-making process. In this paper, we propose a new algorithmic approach based on contextual multi-armed bandit problems (CMAB) to identify and counteract such biased expertise. We explore homogeneous, heterogeneous and polarised expert groups and show that this approach is able to effectively exploit the collective expertise, outperforming state-of-the-art methods, especially when the quality of the provided expertise degrades. Our novel CMAB-inspired approach achieves a higher final performance and does so while converging more rapidly than previous adaptive algorithms.