mlsanityData to 14 Oct 2026

Invariance-Based Dynamic Regret Minimization

arXiv 2603.03843 ↗First posted 4 Mar 2026stat.ML cs.LGcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

We consider stochastic non-stationary linear bandits where the linear parameter connecting contexts to the reward changes over time. Existing algorithms in this setting localize the policy by gradually discarding or down-weighting past data, effectively shrinking the time horizon over which learning can occur. However, in many settings historical data may still carry partial information about the reward model. We propose to leverage such data while adapting to changes, by assuming the reward model decomposes into stationary and non-stationary components. Based on this assumption, we introduce ISD-linUCB, an algorithm that uses past data to learn invariances in the reward model and subsequently exploits them to improve online performance. We show both theoretically and empirically that leveraging invariance reduces the problem dimensionality, yielding significant regret improvements in fast-changing environments when sufficient historical data is available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 9 of its 21 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

9 of 21 references.

References · 9

  1. 2019Weighted Linear Bandits for Non-Stationary Environmentscited by 86
  2. 2018Learning to Optimize under Non-Stationaritycited by 85
  3. 2021A Simple Approach for Non-stationary Linear Banditscited by 54
  4. 2021Leveraging Good Representations in Linear Contextual Banditscited by 29
  5. 2021Invariant Policy Learning: A Causal Perspectivecited by 17
  6. 2022Non-Stationary Representation Learning in Sequential Linear Banditscited by 13
  7. 2024Meta Learning in Bandits within Shared Affine Subspacescited by 4
  8. 2024Non-Stationary Latent Auto-Regressive Banditscited by 3
  9. 2024Invariant Subspace Decompositioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Dynamic Regret for Non-Stationary Linear Bandits via Misspecification Reductions
  2. 2026Flow-Corrected Thompson Sampling for Non-Stationary Contextual Bandits
  3. 2023Reward Learning as Doubly Nonparametric Bandits: Optimal Design and Scaling Laws
  4. 2026Offline-to-Online Learning in Linear Bandits
  5. 2026Bellman-Centric Learning: Near-Optimal Regret for Linear Bandits with Memory
  6. 2026Nonstationary Generalized Linear Bandits with Discounted Online Mirror Descent
  7. 2026Dynamic Minimax Regret Optimization for Robust LLM Post-Training
  8. 2021Adapting to Misspecification in Contextual Bandits with Offline Regression Oracles