mlsanityData to 7 Oct 2026

ReBRAC-v2: The Return of the King

arXiv 2608.01205 ↗First posted 2 Aug 2026cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Recent offline reinforcement learning methods increasingly rely on expressive generative policies and specialized value-guidance mechanisms. We ask whether comparable progress can instead come from systematically modernizing a conventional behavior-regularized actor-critic while preserving its algorithmic simplicity. We introduce ReBRAC-v2, which directly trains an exact-likelihood normalizing flow as the RL actor, combines likelihood, MSE, and MAE behavior regularization, and integrates a classification-based residual critic, staged optimization, and multi-sample test-time action selection. Rather than tuning this recipe separately for every task, we develop a single shared configuration via roughly 600 Bayesian proposals on six challenging OGBench tasks, freeze all structural and optimization choices, and adapt only two behavior-regularization coefficients over a 16-point grid. Across ten common state-based OGBench categories, ReBRAC-v2 averages 74.8 compared to 52.3 for the next-best aggregate result and ranks first in eight categories. The same recipe, without structural changes, obtains the strongest averages in our comparisons on D4RL AntMaze (90.2) and Adroit (33.6). Fixed-recipe ablations show the largest sensitivity to the selected mixed cloning objective, staged training, sufficient flow capacity, and multi-sample inference, while showing that several smaller choices depend on the values of other hyperparameters. These results show that disciplined, transferable engineering can achieve state-of-the-art aggregate performance without abandoning a minimalist offline RL foundation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 27 of its 32 references that are in the map. First posted on arXiv 2 Aug 2026.

Builds on

27 of 32 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024The Role of Deep Learning Regularizations on Actors in Offline RL11 Sep
  2. 2024Is Value Learning Really the Main Bottleneck in Offline RL?13 Jun
  3. 2022Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizes28 Nov
  4. 2022CORL: Research-oriented Deep Offline Reinforcement Learning Library13 Oct
  5. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  6. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble4 Oct
  7. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  8. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  9. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr
  10. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 17

  1. 2016Layer Normalizationcited by 5,579
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2015Variational Inference with Normalizing Flowscited by 2,161
  4. 2016Density estimation using Real NVPcited by 2,029
  5. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  6. 2019Neural Spline Flowscited by 590
  7. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 216
  8. 2022Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Modelscited by 172
  9. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 136
  10. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 108
  11. 2018Improving Regression Performance with Distributional Lossescited by 66
  12. 2022Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flowscited by 21
5 more
  1. 2023Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregressioncited by 14
  2. 2023Learning to Optimize for Reinforcement Learningcited by 12
  3. 2025NinA: Normalizing Flows in Action. Training VLA Models with Normalizing Flowscited by 5
  4. 2026Mining Generalizable Activation Functionscited by 3
  5. 2026SERNF: Sample-Efficient Real-World Dexterous Policy Fine-Tuning via Action-Chunked Critics and Normalizing Flowscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Revisiting the Minimalist Approach to Offline Reinforcement Learning
  2. 2026Decoupling Policy Extraction for Offline Reinforcement Learning
  3. 2021BRAC+: Improved Behavior Regularized Actor Critic for Offline Reinforcement Learning
  4. 2025B3C: A Minimalist Approach to Offline Multi-Agent Reinforcement Learning
  5. 2026Reinforcement Learning via Value Gradient Flow
  6. 2025Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RL
  7. 2025A Clean Slate for Offline Reinforcement Learning
  8. 2026Simple Actors and Deep Critics for Scalable Reinforcement Learning