mlsanityData to 14 Oct 2026

Learning Control Policies for Variable Objectives from Offline Data

arXiv 2308.06127 ↗First posted 11 Aug 2023cs.LGcited by 3

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Offline reinforcement learning provides a viable approach to obtain advanced control strategies for dynamical systems, in particular when direct interaction with the environment is not available. In this paper, we introduce a conceptual extension for model-based policy search methods, called variable objective policy (VOP). With this approach, policies are trained to generalize efficiently over a variety of objectives, which parameterize the reward function. We demonstrate that by altering the objectives passed as input to the policy, users gain the freedom to adjust its behavior or re-balance optimization targets at runtime, without need for collecting additional observation batches or re-training.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 38 references that are in the map. First posted on arXiv 11 Aug 2023.

Builds on

23 of 38 references.

Closest ancestors · 13

Same direction, newest first

  1. 2022Confidence-Conditioned Value Functions for Offline Reinforcement Learning8 Dec
  2. 2022User-Interactive Offline Reinforcement Learning21 May
  3. 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learning26 Apr
  4. 2022Comparing Model-free and Model-based Algorithms for Offline Reinforcement Learning14 Jan
  5. 2021COMBO: Conservative Offline Model-Based Policy Optimization16 Feb
  6. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning1 Feb
  7. 2020Overcoming Model Bias for Robust Offline Deep Reinforcement Learning12 Aug
  8. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  9. 2020MOPO: Model-based Offline Policy Optimization27 May
  10. 2020MOReL : Model-Based Offline Reinforcement Learning12 May
  11. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  12. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr
1 more
  1. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 10

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2016OpenAI Gymcited by 1,736
  3. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  4. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  5. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  6. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  7. 2018Temporal Difference Models: Model-Free Deep RL for Model-Based Controlcited by 89
  8. 2016Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networkscited by 49
  9. 2017A Benchmark Environment Motivated by Industrial Control Problemscited by 31
  10. 2017Batch Reinforcement Learning on the Industrial Benchmark: First Experiencescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Why long model-based rollouts are no reason for bad Q-value estimatescited by 1
  2. 2025Reward Dimension Reduction for Scalable Multi-Objective Reinforcement Learningcited by 1
  3. 2026Constrained Reinforcement Learning Using Successor Representationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022A Unified Framework for Alternating Offline Model Training and Policy Learning
  2. 2022Latent-Variable Advantage-Weighted Policy Optimization for Offline RL
  3. 2024Offline Hierarchical Reinforcement Learning via Inverse Optimization
  4. 2022Offline Reinforcement Learning with Closed-Form Policy Improvement Operators
  5. 2022Flow to Control: Offline Reinforcement Learning with Lossless Primitive Discovery
  6. 2023Equivariant Data Augmentation for Generalization in Offline Reinforcement Learning
  7. 2023Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutions
  8. 2024Policy-regularized Offline Multi-objective Reinforcement Learning