# Equivalence testing with data-dependent and post-hoc equivalence margins

arXiv: https://arxiv.org/abs/2603.16213 · HTML: https://mlsanity.com/paper/2603.16213/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2026.

## Built on by (most cited first, 2 of 2)

- [Post-Selection Distributional Model Evaluation](https://mlsanity.com/paper/2603.23055/) (2026)
- [Bentkus-type asymptotic e-values](https://mlsanity.com/paper/2606.06332/) (2026)

## Abstract (arXiv)

Equivalence testing compares the hypothesis that an effect μ is large against the alternative that it is negligible. Here, `large' is classically expressed as being larger than some `equivalence margin' Delta. A longstanding problem is that this margin must be specified but can rarely be objectively justified in practice. We lay the foundation for an alternative paradigm, arguing to instead report a data-dependent margin widehatDeltaα that bounds the true effect μ with probability 1 - α. Our key argument is that widehatDeltaα is more useful than a test outcome at a fixed margin Delta, as measured by the guarantees it offers to decision makers. We generalize this to a curve of margins α mapsto widehatDeltaα, uniformly valid under the post-hoc selection of the margin. These ideas rely on e-values, which we derive for models that are strictly totally positive of order 3, nesting the classical z-test and t-test settings.