mlsanityData to 14 Oct 2026

Towards an Adaptive Social Game-Playing Robot: An Offline Reinforcement Learning-Based Framework

arXiv 2509.16858 ↗First posted 21 Sep 2025cs.ROcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

HRI research increasingly demands robots that go beyond task execution to respond meaningfully to user emotions. This is especially needed when supporting students with learning difficulties in game-based learning scenarios. Here, the objective of these robots is to train users with game-playing skills, and this requires robots to get input about users' interests and engagement. In this paper, we present a system for an adaptive social game-playing robot. However, creating such an agent through online RL requires extensive real-world training data and potentially be uncomfortable for users. To address this, we investigate offline RL as a safe and efficient alternative. We introduce a system architecture that integrates multimodal emotion recognition and adaptive robotic responses. We also evaluate the performance of various offline RL algorithms using a dataset collected from a real-world human-robot game-playing scenario. Our results indicate that BCQ and DDQN offer the greatest robustness to hyperparameter variations, whereas CQL is the most effective at mitigating overestimation bias. Through this research, we aim to inform the selection and design of reliable offline RL policies for real-world social robotics. Ultimately, this work provides a foundational step toward creating socially intelligent agents that can learn complex and emotion-adaptive behaviors entirely from offline datasets, ensuring both human comfort and practical scalability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 7 of its 23 references that are in the map. First posted on arXiv 21 Sep 2025.

Builds on

7 of 23 references.

Closest ancestors · 3

Same direction, newest first

  1. 2021d3rlpy: An Offline Deep Reinforcement Learning Library6 Nov
  2. 2020Hyperparameter Selection for Offline Reinforcement Learning17 Jul
  3. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun

Other references · 4

  1. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  2. 2019Soft Actor-Critic for Discrete Action Settingscited by 174
  3. 2022Offline Reinforcement Learning for Safer Blood Glucose Control in People with Type 1 Diabetescited by 23
  4. 2017Applying the Wizard-of-Oz Technique to Multimodal Human-Robot Dialoguecited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
  2. 2024RLHF Workflow: From Reward Modeling to Online RLHF
  3. 2024Online Behavior Modification for Expressive User Control of RL-Trained Robots
  4. 2021Improving Reinforcement Learning with Human Assistance: An Argument for Human Subject Studies with HIPPO Gym
  5. 2025Behavior-Adaptive Q-Learning: A Unifying Framework for Offline-to-Online RL
  6. 2022Real World Offline Reinforcement Learning with Realistic Data Source
  7. 2022Offline Reinforcement Learning for Human-Guided Human-Machine Interaction with Private Information
  8. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learning