mlsanityData to 14 Oct 2026

Variational OOD State Correction for Offline Reinforcement Learning

arXiv 2505.00503 ↗First posted 1 May 2025cs.LG cs.AI cs.ROcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

The performance of Offline reinforcement learning is significantly impacted by the issue of state distributional shift, and out-of-distribution (OOD) state correction is a popular approach to address this problem. In this paper, we propose a novel method named Density-Aware Safety Perception (DASP) for OOD state correction. Specifically, our method encourages the agent to prioritize actions that lead to outcomes with higher data density, thereby promoting its operation within or the return to in-distribution (safe) regions. To achieve this, we optimize the objective within a variational framework that concurrently considers both the potential outcomes of decision-making and their density, thus providing crucial contextual information for safe decision-making. Finally, we validate the effectiveness and feasibility of our proposed method through extensive experimental evaluations on the offline MuJoCo and AntMaze suites.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 6 of its 33 references that are in the map. First posted on arXiv 1 May 2025.

Builds on

6 of 33 references.

Closest ancestors · 1

Same direction, newest first

  1. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 5

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  4. 2015Importance Weighted Autoencoderscited by 456
  5. 2016Tutorial on Variational Autoencoderscited by 409

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Offline Reinforcement Learning with OOD State Correction and OOD Action Suppression
  2. 2023Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning
  3. 2024ODRL: A Benchmark for Off-Dynamics Reinforcement Learning
  4. 2024Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning
  5. 2024Towards Fast Safe Online Reinforcement Learning via Policy Finetuning
  6. 2026Generative OOD-regularized Model-based Policy Optimization
  7. 2021OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation
  8. 2024FAWAC: Feasibility Informed Advantage Weighted Regression for Persistent Safety in Offline Reinforcement Learning