mlsanityData to 7 Oct 2026

Gated Decoupled Compositional Bandits: A Unified Theory of Contextual Bandits with Supervised-Calibrated Action Scaling and Pre-Execution Gating

arXiv 2608.21993 ↗First posted 22 Aug 2026cs.LGcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

We introduce Gated Decoupled Compositional Bandits (GDCB), a family of contextual bandit algorithms with three structural innovations that jointly fall outside the taxonomy of LinUCB, LinTS, HierTS, factored bandits, neural contextual bandits, and RLHF. In a GDCB system: (i) the action delivered to the environment is the composition of a nominal arm, drawn by a discrete or hierarchical bandit, with a context-dependent scaler; (ii) the scaler parameter is learned in a separate supervised loop, not jointly with arm selection; and (iii) every action passes through a pre-execution gate that may modify or veto the composed action before it reaches the environment. We formalise this class of algorithms, prove four structural theorems characterising its statistical behaviour, and show that six industrially significant systems -- short-term rental dynamic pricing, clinical drug dosing, credit origination, grid demand response, content moderation, and LLM tool-use agents -- are all instances of GDCB, differing only in the composition operator, scaler family, and gate. The central result is the Decoupling Variance Reduction theorem: a well-calibrated scaler removes context-induced variance from the arm-to-reward mapping, turning a non-stationary bandit problem into an approximately stationary one. The Gate-Induced Equivalence theorem shows that under a stationary gate, historical data collected under any prior policy is a valid warm-up initialiser without importance-sampling correction, generalising the companion P-HITL result (arXiv:2606.02595) from human approval to arbitrary gates. In regulated, high-stakes domains, constraints usually treated as deployment frictions -- approval gates, compliance rules, safety shields -- are the mechanism that makes fast deployment possible, not an obstacle to it. The companion paper validates instance 1 (STR dynamic pricing) on real production data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 14 of its 56 references that are in the map. First posted on arXiv 22 Aug 2026.

Builds on

14 of 56 references.

Closest ancestors · 1

Same direction, newest first

  1. 2017Deep reinforcement learning from human preferences12 Jun

Other references · 13

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  4. 2017Safe Reinforcement Learning via Shieldingcited by 427
  5. 2013Learning to Optimize Via Posterior Samplingcited by 253
  6. 2016Causal Bandits: Learning Good Interventions via Causal Inferencecited by 100
  7. 2019Linear Stochastic Bandits Under Safety Constraintscited by 90
  8. 2019Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activitycited by 70
  9. 2021Metadata-based Multi-Task Bandits with Bayesian Hierarchical Modelscited by 24
  10. 2022Deep Hierarchy in Banditscited by 18
  11. 2021Leveraging Expert Consistency to Improve Algorithmic Decision Supportcited by 17
  12. 2018Factored Banditscited by 7
1 more
  1. 2026Human-in-the-Loop Contextual Bandits for Short-Term Rental Dynamic Pricing: Structural Equivalence of Historical Warm-Up and Approval-Gated Live Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Flexible and Efficient Contextual Bandits with Heterogeneous Treatment Effect Oracles
  2. 2026Flow-Corrected Thompson Sampling for Non-Stationary Contextual Bandits
  3. 2021Adapting to Misspecification in Contextual Bandits with Offline Regression Oracles
  4. 2026RIE-Greedy: Regularization-Induced Exploration for Contextual Bandits
  5. 2024Diffusion Models Meet Contextual Bandits
  6. 2021Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization
  7. 2020Tractable contextual bandits beyond realizability
  8. 2025HD-CB: The First Exploration of Hyperdimensional Computing for Contextual Bandits Problems