mlsanityData to 14 Oct 2026

Arithmetic Actor Heads and Training Stabilization for Out-of-Distribution Reinforcement Learning

arXiv 2610.05143 ↗First posted 4 Oct 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations also fits: RL theory and sample complexity

Abstract (arXiv)

Reinforcement learning (RL) policies can deteriorate under out-of-distribution (OOD) magnitude shifts. Starting from soft actor-critic (SAC) and its Bayesian Amnesic Piecewise-Robust (BAPR) predecessor, we study the causal-symbolic BAPR (CS-BAPR) family. The practical method combines six training-stabilization settings with alternative actor heads: a Neural Addition Unit (NAU) with a Neural Multiplication Unit (NMU)-inspired quadratic correction, a Kolmogorov-Arnold Network (KAN), or a multilayer perceptron (MLP) with rectified linear unit (ReLU) or hyperbolic-tangent activations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 28 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

18 of 28 references.

References · 18

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2019Invariant Risk Minimizationcited by 1,854
  3. 2024KAN: Kolmogorov-Arnold Networkscited by 649
  4. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  5. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 548
  6. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 302
  7. 2021Towards Causal Representation Learningcited by 209
  8. 2018Learning Equations for Extrapolation and Controlcited by 156
  9. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 109
  10. 2018Neural Arithmetic Logic Unitscited by 70
  11. 2023A Survey on Causal Reinforcement Learningcited by 45
  12. 2023Natural Actor-Critic for Robust Reinforcement Learning with Function Approximationcited by 36
6 more
  1. 2020Neural Arithmetic Unitscited by 28
  2. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 26
  3. 2025Single Agent Robust Deep Reinforcement Learning for Bus Fleet Controlcited by 4
  4. 2026RE-SAC: Disentangling aleatoric and epistemic risks in bus fleet control: A stable and robust ensemble DRL approachcited by 2
  5. 2025On the failure of ReLU activation for physics-informed machine learningcited by 1
  6. 2026BAPR: Bayesian amnesic piecewise-robust reinforcement learning for non-stationary continuous controlcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025DR-SAC: Distributionally Robust Soft Actor-Critic for Reinforcement Learning under Uncertainty
  2. 2025SACn: Soft Actor-Critic with n-step Returns
  3. 2025Bidirectional Soft Actor-Critic: Leveraging Forward and Reverse KL Divergence for Efficient Reinforcement Learning
  4. 2022Adversarially Trained Actor Critic for Offline Reinforcement Learning
  5. 2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning
  6. 2024AFU: Actor-Free critic Updates in off-policy RL for continuous control
  7. 2024Improving Actor-Critic Training with Steerable Action-Value Approximation Errors
  8. 2024Risk-Sensitive Soft Actor-Critic for Robust Deep Reinforcement Learning under Distribution Shifts