mlsanityData to 14 Oct 2026

Reinforcement Learning Measurement Model

arXiv 2605.09305 ↗First posted 10 May 2026stat.ME cs.HC cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Interactive assessments generate sequential process data that are not well handled by conventional item response models. Existing MDP-based measurement approaches, such as the Markov decision process measurement model (MDP-MM, LaMar, 2018), link action choices to state-action values, but their reliance on person-specific tabular value functions makes them difficult to scale beyond small, fully enumerated tasks. We propose the Reinforcement Learning Measurement Model (RLMM), a measurement framework that decouples person-level choice sensitivity from task-level value representation through a shared parametric action-value function, making estimation more computationally efficient for larger process-data settings. The model combines a Boltzmann choice rule with normalized advantages, a soft Bellman consistency penalty, and a block-coordinate MAP procedure for joint estimation, while also yielding step-level influence diagnostics for identifying behaviorally critical decisions. In peg-solitaire simulations, the RLMM achieved higher estimation accuracy and substantially lower runtime than the original MDP-MM, with advantages increasing as task complexity grew. In AQUALAB gameplay logs, the estimated person parameter was positively associated with cumulative reward, task completion, and behavioral efficiency. These results show that the RLMM extends decision-process-based psychometric models to larger and more behaviorally realistic environments while preserving an interpretable latent trait tied to decision making steps.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 1 of its 28 references that are in the map. First posted on arXiv 10 May 2026.

Builds on

1 of 28 references.

References · 1

  1. 2019A Continuous-Time Dynamic Choice Measurement Model for Problem-Solving Process Datacited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Statistical and Algorithmic Foundations of Reinforcement Learning
  2. 2024Secrets of RLHF in Large Language Models Part II: Reward Modeling
  3. 2026Reinforcement Learning from Human Feedback: A Statistical Perspective
  4. 2025Policy-labeled Preference Learning: Is Preference Enough for RLHF?
  5. 2025Reinforcement Learning from Human Feedback
  6. 2024Towards Comprehensive Preference Data Collection for Reward Modeling
  7. 2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning
  8. 2025Generalized Linear Markov Decision Process