# Eliciting Risk Aversion with Inverse Reinforcement Learning via Interactive Questioning

arXiv: https://arxiv.org/abs/2308.08427 · HTML: https://mlsanity.com/paper/2308.08427/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 42 references that are in the map.
- First posted on arXiv 16 Aug 2023.

## Builds on (most cited first, 15 of 15)

- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [Identifiability in inverse reinforcement learning](https://mlsanity.com/paper/2106.03498/) (2021)
- [Invariance in Policy Optimisation and Partial Identifiability in Reward Learning](https://mlsanity.com/paper/2203.07475/) (2022)
- [Repeated Inverse Reinforcement Learning](https://mlsanity.com/paper/1705.05427/) (2017)
- [Towards Resolving Unidentifiability in Inverse Reinforcement Learning](https://mlsanity.com/paper/1601.06569/) (2016)
- [Risk-sensitive Inverse Reinforcement Learning via Semi- and Non-Parametric Methods](https://mlsanity.com/paper/1711.10055/) (2017)
- [Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal control](https://mlsanity.com/paper/2109.08947/) (2021)
- [Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning](https://mlsanity.com/paper/2206.14666/) (2022)
- [Learning from an Exploring Demonstrator: Optimal Reward Estimation for Bandits](https://mlsanity.com/paper/2106.14866/) (2021)
- [Inverse Optimization of Convex Risk Functions](https://mlsanity.com/paper/1607.07099/) (2016)
- [Interactive Inverse Reinforcement Learning for Cooperative Games](https://mlsanity.com/paper/2111.04698/) (2021)
- [Environment Design for Inverse Reinforcement Learning](https://mlsanity.com/paper/2210.14972/) (2022)
- [Risk-Averse Markov Decision Processes through a Distributional Lens](https://mlsanity.com/paper/2203.09612/) (2022)
- [Active Learning for Risk-Sensitive Inverse Reinforcement Learning](https://mlsanity.com/paper/1909.07843/) (2019)

## Built on by (most cited first, 2 of 2)

- [Distributional Inverse Reinforcement Learning](https://mlsanity.com/paper/2510.03013/) (2025)
- [A Noise-Robust Elicit-to-Optimize Framework for Distortion Riskmetrics via Inverse Reinforcement Learning](https://mlsanity.com/paper/2607.14373/) (2026)

## Abstract (arXiv)

We investigate a framework for robo-advisors to estimate non-expert clients' risk aversion using adaptive binary-choice questionnaires. We model risk aversion using cost functions and spectral risk measures in a static setting. We prove the finite-sample identifiability and, for properly designed questions, obtain a convergence rate of √N up to a logarithmic factor, where N is the number of questions. We introduce the notion of distinguishing power and demonstrate, through simulated experiments, that designing questions by maximizing distinguishing power achieves satisfactory accuracy in learning risk aversion with fewer than 50 questions. We also provide a preliminary investigation of an infinite-horizon setting with an additional discount factor for dynamic risk aversion, establishing qualitative identifiability in this case.