Data to 14 Oct 2026

Enhancing Human-Likeness in Reinforcement Learning Agents via Hierarchical Macro Action Quantization

arXiv 2605.30928 ↗First posted 29 May 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Human-like agents are a long-standing goal of artificial intelligence. Despite strong performance, most reinforcement learning (RL) agents remain reward-driven and often exhibit behaviors that differ from humans, limiting interpretability and reliability. In this work, we introduce a novel human-like RL framework that predicts action sequences closely aligned with human behaviors while maximizing rewards. Specifically, we encode human demonstrations into macro actions using a hierarchical macro action quantization approach (HiMAQ) consisting of two successive levels of vector quantization. The lower quantization level maps input actions to fine-grained subaction clusters, while the higher quantization level aggregates these subaction clusters into action clusters. Extensive evaluations on the D4RL benchmarks show that our hierarchical approach outperforms the non-hierarchical baseline (MAQ), achieving higher human-likeness scores and better success rates than previous RL agents. The improvements generalize across integrations with various RL algorithms, namely IQL, SAC, and RLPD.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 27 of its 36 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

27 of 36 references.

References · 27

2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
2017Neural Discrete Representation Learningcited by 4,712
2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
2016End to End Learning for Self-Driving Carscited by 1,009
2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
2022Temporal Difference Learning for Model Predictive Controlcited by 397
2023Efficient Online Reinforcement Learning with Offline Datacited by 363
2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 298
2023Imitating Human Behaviour with Diffusion Modelscited by 263

15 more

2025Flow Q-Learningcited by 184
2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 148
2020Regularizing Action Policies for Smooth Control with Reinforcement Learningcited by 59
2021Vector Quantized Models for Planningcited by 48
2023Action-Quantized Offline Reinforcement Learning for Robotic Skill Learningcited by 36
2024Hierarchical Vector Quantization for Unsupervised Action Segmentationcited by 19
2021Navigation Turing Test (NTT): Learning to Evaluate Human-Like Navigationcited by 17
2016Deep Reinforcement Learning With Macro-Actionscited by 14
2025Action Tokenizer Matters in In-Context Imitation Learningcited by 14
2022Revisiting Discrete Soft Actor-Criticcited by 13
2023Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Gamescited by 11
2024Gradient-based Regularization for Action Smoothness in Robotic Control with Reinforcement Learningcited by 10
2019Flight Controller Synthesis Via Deep Reinforcement Learningcited by 5
2023Towards Human-Like RL: Taming Non-Naturalistic Behavior in Deep RL via Adaptive Behavioral Costs in 3D Gamescited by 4
2025Learning Human-Like RL Agents Through Trajectory Optimization With Action Quantizationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way