Clone-Robust Weights in Metric Spaces: Handling Redundancy Bias for Benchmark Aggregation
Not on the map. Why?
Abstract (arXiv)
We are given a set of elements in a metric space. The distribution of the elements is arbitrary, possibly adversarial. Can we weigh the elements in a way that is resistant to such (adversarial) manipulations? This problem arises in various contexts. For instance, the elements could represent data points, requiring robust domain adaptation. Alternatively, they might represent tasks to be aggregated into a benchmark; or questions about personal political opinions in voting advice applications. This article introduces a theoretical framework for dealing with such problems. We propose clone-proof weighting functions as a solution concept. These functions distribute importance across elements of a set such that similar objects (``clones'') share (some of) their weights, thus avoiding a potential bias introduced by their multiplicity. Our framework extends the maximum uncertainty principle to accommodate general metric spaces and includes a set of axioms -- symmetry, continuity, and clone-proofness -- that guide the construction of weighting functions. Finally, we address the existence of weighting functions satisfying our axioms in the significant case of Euclidean spaces and propose a general method for their construction.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 69 references that are in the map. First posted on arXiv 5 Feb 2025.
Builds on
27 of 69 references.
References · 27
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
- 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
- 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
- 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
- 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
- 2018Neural Network Acceptability Judgmentscited by 952
- 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
- 2018Deep Visual Domain Adaptation: A Surveycited by 634
- 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
- 2019Task2Vec: Task Embedding for Meta-Learningcited by 197
- 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
- 2020Geometric Dataset Distances via Optimal Transportcited by 157
15 more
- 2017Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samplescited by 147
- 2020Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspectivecited by 143
- 2017Class Rectification Hard Mining for Imbalanced Deep Learningcited by 67
- 2018Re-evaluating Evaluationcited by 47
- 2021Better than Average: Paired Evaluation of NLP Systemscited by 30
- 2017Fine-grained Recognition in the Wild: A Multi-Task Domain Adaptation Approachcited by 27
- 2024Inherent Trade-Offs between Diversity and Stability in Multi-Task Benchmarkscited by 26
- 2020Domain2Vec: Domain Embedding for Unsupervised Domain Adaptationcited by 24
- 2022Wasserstein Task Embedding for Measuring Task Similaritiescited by 24
- 2021NeuroBack: Improving CDCL SAT Solving using Graph Neural Networkscited by 19
- 2022Vote'n'Rank: Revision of Benchmarking with Social Choice Theorycited by 15
- 2023Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarkscited by 14
- 2021How not to Lie with a Benchmark: Rearranging NLP Leaderboardscited by 9
- 2022Spectral, Probabilistic, and Deep Metric Learning: Tutorial and Surveycited by 9
- 2022Sybil-Proof Diffusion Auction in Social Networkscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026AI Alignment From Social Choice Perspectivescited by 0
- 2026AI Alignment From Social Choice Perspectives19 Jun 2026