mlsanityData to 14 Oct 2026

Universal Sequence Preconditioning

arXiv 2502.06545 ↗First posted 10 Feb 2025cs.LG stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

We study the problem of preconditioning in sequential prediction. From the theoretical lens of linear dynamical systems, we show that convolving the target sequence corresponds to applying a polynomial to the hidden transition matrix. Building on this insight, we propose a universal preconditioning method that convolves the target with coefficients from orthogonal polynomials such as Chebyshev or Legendre. We prove that this approach reduces regret for two distinct prediction algorithms and yields the first ever sublinear and hidden-dimension-independent regret bounds (up to logarithmic factors) that hold for systems with marginally table and asymmetric transition matrices. Finally, extensive synthetic and real-world experiments show that this simple preconditioning strategy improves the performance of a diverse range of algorithms, including recurrent neural networks, and generalizes to signals beyond linear dynamical systems.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 23 of its 46 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

23 of 46 references.

References · 23

  1. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  2. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  3. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  4. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  5. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  6. 2019Introduction to Online Convex Optimizationcited by 803
  7. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  8. 2017Learned Optimizers that Scale and Generalizecited by 110
  9. 2019Online Control with Adversarial Disturbancescited by 92
  10. 2018Regret Bounds for Robust Adaptive Control of the Linear Quadratic Regulatorcited by 86
  11. 2020Improper Learning for Non-Stochastic Controlcited by 83
  12. 2019Learning Linear-Quadratic Regulators Efficiently with only √T Regretcited by 70
11 more
  1. 2018Online Linear Quadratic Controlcited by 56
  2. 2017Learning Linear Dynamical Systems via Spectral Filteringcited by 46
  3. 2020Black-Box Control for Linear Dynamical Systemscited by 43
  4. 2019Logarithmic Regret for Online Controlcited by 38
  5. 2018Spectral Filtering for General Linear Dynamical Systemscited by 37
  6. 2013Online Learning for Time Series Predictioncited by 33
  7. 2023A New Approach to Learning Linear Dynamical Systemscited by 23
  8. 2023Spectral State Space Modelscited by 21
  9. 2020No-Regret Prediction in Marginally Stable Systemscited by 18
  10. 2018On-Line Learning of Linear Dynamical Systems: Exponential Forgetting in Kalman Filterscited by 16
  11. 2024FutureFill: Fast Generation from Convolutional Sequence Modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

  1. 2025Research Program: Theory of Learning in Dynamical Systemscited by 5
  2. 2025A New Approach to Controlling Linear Dynamical Systemscited by 2
  3. 2026Spectral Filtering for Complex Linear Dynamical Systemscited by 2
  4. 2025SpectraLDS: Provable Distillation for Linear Dynamical Systemscited by 1
  5. 2026The Power of Second Order Methods for Sequence Preconditioningcited by 1
  6. 2026Online Nonstochastic Prediction: Logarithmic Regret via Predictive Online Least Squarescited by 0