The Price of Tolerance in Distribution Testing
Not on the map. Why?
Abstract (arXiv)
We revisit the problem of tolerant distribution testing. That is, given samples from an unknown distribution p over 1, dots, n, is it varepsilon1-close to or varepsilon2-far from a reference distribution q (in total variation distance)? Despite significant interest over the past decade, this problem is well understood only in the extreme cases. In the noiseless setting (i.e., varepsilon1 = 0) the sample complexity is Theta(√n), strongly sublinear in the domain size. At the other end of the spectrum, when varepsilon1 = varepsilon2/2, the sample complexity jumps to the barely sublinear Theta(n/log n). However, very little is known about the intermediate regime. We fully characterize the price of tolerance in distribution testing as a function of n, varepsilon1, varepsilon2, up to a single log n factor. Specifically, we show the sample complexity to be [tilde Thetaleft(frac√nvarepsilon22 + fracnlog n · max leftfracvarepsilon1varepsilon22,left(fracvarepsilon1varepsilon22right)!!2rightright),] providing a smooth tradeoff between the two previously known cases. We also provide a similar characterization for the problem of tolerant equivalence testing, where both p and q are unknown. Surprisingly, in both cases, the main quantity dictating the sample complexity is the ratio varepsilon1/varepsilon22, and not the more intuitive varepsilon1/varepsilon2. Of particular technical interest is our lower bound framework, which involves novel approximation-theoretic tools required to handle the asymmetry between varepsilon1 and varepsilon2, a challenge absent from previous works.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 9 of its 50 references that are in the map. First posted on arXiv 25 Jun 2021.
Builds on
9 of 50 references.
References · 9
- 2013Optimal Algorithms for Testing Closeness of Discrete Distributionscited by 37
- 2015Optimal Testing for Properties of Distributionscited by 36
- 2014Testing Identity of Structured Distributionscited by 19
- 2017Optimal Identity Testing with High Probabilitycited by 19
- 2020Optimal Testing of Discrete Distributions with High Probabilitycited by 15
- 2016Collision-based Testers are Optimal for Uniformity and Closenesscited by 14
- 2017Which Distribution Distances are Sublinearly Testable?cited by 14
- 2017Hypothesis Testing for High-Dimensional Multinomials: A Selective Reviewcited by 7
- 2015Testing Closeness With Unequal Sized Samplescited by 4
Built on by
Citing papers per year: 2022: 2; 2023: 1; 2024: 4; 2025: 4; 2026: 2.
13 papers.
Sort:
- 2024Tolerant Algorithms for Learning with Arbitrary Covariate Shiftcited by 15
- 2022PAC Verification of Statistical Algorithmscited by 8
- 2023Testing Closeness of Multivariate Distributions via Ramsey Theorycited by 8
- 2024Replicable Uniformity Testingcited by 4
- 2024Optimal Algorithms for Augmented Testing of Discrete Distributionscited by 2
- 2025Replicable Distribution Testingcited by 2
- 2022Mathematical Framework for Online Social Media Auditingcited by 1
- 2025Zero-Shot Attribution for Large Language Models: A Distribution Testing Approachcited by 1
- 2024Reframing Data Value for Large Language Models Through the Lens of Plausibilitycited by 0
- 2025Are Two Datasets Close Enough With Statistical Significance? A Kernel Distributional Closeness Testing Approachcited by 0
- 2025Dimension Agnostic Testing of Survey Data Credibility through the Lens of Regressioncited by 0
- 2026Differentially Private Verification of Distribution Propertiescited by 0
1 more
- 2026When Are Trade-Off Functions Testable from Finite Samples?11 May 2026
- 2026Differentially Private Verification of Distribution Properties12 Apr 2026
- 2025Dimension Agnostic Testing of Survey Data Credibility through the Lens of Regression28 Aug 2025
- 2025Are Two Datasets Close Enough With Statistical Significance? A Kernel Distributional Closeness Testing Approach17 Jul 2025
- 2025Replicable Distribution Testing3 Jul 2025
- 2025Zero-Shot Attribution for Large Language Models: A Distribution Testing Approach25 Jun 2025
- 2024Optimal Algorithms for Augmented Testing of Discrete Distributions1 Dec 2024
- 2024Replicable Uniformity Testing12 Oct 2024
- 2024Reframing Data Value for Large Language Models Through the Lens of Plausibility30 Aug 2024
- 2024Tolerant Algorithms for Learning with Arbitrary Covariate Shift4 Jun 2024
- 2023Testing Closeness of Multivariate Distributions via Ramsey Theory22 Nov 2023
- 2022PAC Verification of Statistical Algorithms28 Nov 2022
1 more
- 2022Mathematical Framework for Online Social Media Auditing12 Sep 2022
- 2024Tolerant Algorithms for Learning with Arbitrary Covariate Shiftcited by 15
- 2022PAC Verification of Statistical Algorithmscited by 8
- 2023Testing Closeness of Multivariate Distributions via Ramsey Theorycited by 8
- 2024Replicable Uniformity Testingcited by 4
- 2024Optimal Algorithms for Augmented Testing of Discrete Distributionscited by 2
- 2025Replicable Distribution Testingcited by 2
- 2022Mathematical Framework for Online Social Media Auditingcited by 1
- 2025Zero-Shot Attribution for Large Language Models: A Distribution Testing Approachcited by 1
- 2024Reframing Data Value for Large Language Models Through the Lens of Plausibilitycited by 0
- 2025Are Two Datasets Close Enough With Statistical Significance? A Kernel Distributional Closeness Testing Approachcited by 0
- 2025Dimension Agnostic Testing of Survey Data Credibility through the Lens of Regressioncited by 0
- 2026Differentially Private Verification of Distribution Propertiescited by 0
12 most cited of 13 All 13 (JSON)
- 2026When Are Trade-Off Functions Testable from Finite Samples?11 May 2026
- 2026Differentially Private Verification of Distribution Properties12 Apr 2026
- 2025Dimension Agnostic Testing of Survey Data Credibility through the Lens of Regression28 Aug 2025
- 2025Are Two Datasets Close Enough With Statistical Significance? A Kernel Distributional Closeness Testing Approach17 Jul 2025
- 2025Replicable Distribution Testing3 Jul 2025
- 2025Zero-Shot Attribution for Large Language Models: A Distribution Testing Approach25 Jun 2025
- 2024Optimal Algorithms for Augmented Testing of Discrete Distributions1 Dec 2024
- 2024Replicable Uniformity Testing12 Oct 2024
- 2024Reframing Data Value for Large Language Models Through the Lens of Plausibility30 Aug 2024
- 2024Tolerant Algorithms for Learning with Arbitrary Covariate Shift4 Jun 2024
- 2023Testing Closeness of Multivariate Distributions via Ramsey Theory22 Nov 2023
- 2022PAC Verification of Statistical Algorithms28 Nov 2022
12 newest of 13 All 13 (JSON)