A Noise-Robust Elicit-to-Optimize Framework for Distortion Riskmetrics via Inverse Reinforcement Learning
On the map: Offline RL and imitation learning from data → Inverse reinforcement learning
also in 1 more
Abstract (arXiv)
We propose a noise-robust elicit-to-optimize framework that integrates inverse reinforcement learning (IRL) and reinforcement learning (RL) for eliciting agents' risk preferences and optimizing policies under a broad class of risk objectives characterized by distortion riskmetrics. On the elicitation side, we propose an adaptive Bayesian IRL method that infers agents' latent risk objectives from their noisy observed decisions, explicitly allowing agents to take stochastic and suboptimal actions. We establish the existence of a finite set of distinguishing questions that identifies the preferred distortion riskmetric within the candidate class and prove that the convergence rate of the algorithm is of order O(exp(-cm+O(√mlog m))) under general settings, where c>0 is a constant and m denotes the number of algorithm iterations. On the optimization side, we develop a model-free RL algorithm for optimizing policies under conditional distortion riskmetrics. By representing the objective as an integral of the conditional cost quantile function with respect to the distortion function, the method unifies distortion-riskmetric objectives. We optimize diverse risk objectives by extending the Proximal Policy Optimization (PPO) algorithm with policy, value, and quantile neural networks, where the quantile network estimates the full conditional cost quantile function and enables numerical evaluation of general risk objectives. A comprehensive empirical study demonstrates the framework's elicitation accuracy and effectiveness in complex financial environments.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 14 of its 27 references that are in the map. First posted on arXiv 15 Jul 2026.
Builds on
14 of 27 references.
Closest ancestors · 1
Same direction, newest first
Other references · 13
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 337
- 2018Deep Hedgingcited by 108
- 2020Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learningcited by 28
- 2021Reinforcement Learning with Dynamic Convex Risk Measurescited by 21
- 2020Risk-Aware High-level Decisions for Automated Driving at Occluded Intersections with Reinforcement Learningcited by 20
- 2021Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learningcited by 13
- 2024RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashescited by 10
- 2019Robo-advising: Learning Investors' Risk Preferences via Portfolio Choicescited by 7
- 2021Interactive Inverse Reinforcement Learning for Cooperative Gamescited by 7
- 2019Personalized Robo-Advising: Enhancing Investment through Client Interactioncited by 6
- 2023Eliciting Risk Aversion with Inverse Reinforcement Learning via Interactive Questioningcited by 2
1 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Robust Reinforcement Learning with Dynamic Distortion Risk Measures
- 2025Distributional Inverse Reinforcement Learning
- 2019Active Learning for Risk-Sensitive Inverse Reinforcement Learning
- 2025Inverse Reinforcement Learning via Convex Optimization
- 2025Risk-Averse Constrained Reinforcement Learning with Optimized Certainty Equivalents
- 2023Model Selection for Inverse Reinforcement Learning via Structural Risk Minimization
- 2025Offline Reinforcement Learning via Inverse Optimization
- 2026Actor-Critic Algorithm for Dynamic Expectile and CVaR