mlsanityData to 14 Oct 2026

The Impossible Trinity of Time-Series Validation: A Conservation Law among Training Sufficiency, Test Coverage, and Temporal Causality

arXiv 2609.29530 ↗First posted 2 Sep 2026cs.LG q-fin.ST stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Validating a model on a time series asks for three things at once: each training run should use most of the sample (sufficiency), the test sets should together cover most of the sample (coverage), and training data should come before test data (causality). We prove that the three cannot be had together and price each one. Let α be the smallest training fraction over folds, β the fraction of the sample covered by tests, Lambda the fraction of the sample used as training data from the future of a test point, and δ the distance from a test point to the nearest training point in its future. Every scheme on a sample of length T satisfies α+β ≤ 1+Lambda and α+minβ,δ/T ≤ 1, and under β-mixing the leakage bias at a test point is at most 2Mβmix(δ). In words: going beyond the causal frontier α+β=1 requires training on the future; that future data must sit within (1-α)T of a test point; and its harm depends on its distance, not its amount. Hence expanding walk-forward is exactly the Pareto frontier of causal validation, k-fold cross-validation buys the most future data, and purged k-fold with an embargo pays in distance instead, which is cheap when the process forgets quickly but cannot repair the part of causality demanded by non-stationarity. On pure noise, shuffled 5-fold reports an information coefficient of +0.32, while contiguous 5-fold, using the same amount of future data, reports +0.004.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 34 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

7 of 34 references.

References · 7

  1. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  2. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 303
  3. 2022Game-theoretic statistics and safe anytime-valid inferencecited by 155
  4. 2021The Shape of Learning Curves: a Reviewcited by 70
  5. 2020A Theory of Universal Learningcited by 60
  6. 2019Evaluating time series forecasting models: An empirical study on performance estimation methodscited by 53
  7. 2019Minimizers of the Empirical Risk and Risk Monotonicitycited by 11

Built on by

Not cited yet.