# Reinforcement Learning for Durable Algorithmic Recourse

arXiv: https://arxiv.org/abs/2509.22102 · HTML: https://mlsanity.com/paper/2509.22102/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 26 of 26)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Actionable Recourse in Linear Classification](https://mlsanity.com/paper/1809.06514/) (2018)
- [Algorithmic Recourse: from Counterfactual Explanations to Interventions](https://mlsanity.com/paper/2002.06278/) (2020)
- [Multi-Objective Counterfactual Explanations](https://mlsanity.com/paper/2004.11165/) (2020)
- [Algorithmic recourse under imperfect causal knowledge: a probabilistic approach](https://mlsanity.com/paper/2006.06831/) (2020)
- [The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons](https://mlsanity.com/paper/1912.04930/) (2019)
- [Towards Robust and Reliable Algorithmic Recourse](https://mlsanity.com/paper/2102.13620/) (2021)
- [On the Adversarial Robustness of Causal Algorithmic Recourse](https://mlsanity.com/paper/2112.11313/) (2021)
- [Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse](https://mlsanity.com/paper/2203.06768/) (2022)
- [Algorithmic Recourse in the Wild: Understanding the Impact of Data and Model Shifts](https://mlsanity.com/paper/2012.11788/) (2020)
- [On the Trade-Off between Actionable Explanations and the Right to be Forgotten](https://mlsanity.com/paper/2208.14137/) (2022)
- [Improvement-Focused Causal Recourse (ICR)](https://mlsanity.com/paper/2210.15709/) (2022)
- [CARE: Coherent Actionable Recourse based on Sound Counterfactual Explanations](https://mlsanity.com/paper/2108.08197/) (2021)
- [Generating robust counterfactual explanations](https://mlsanity.com/paper/2304.12943/) (2023)
- [Setting the Right Expectations: Algorithmic Recourse Over Time](https://mlsanity.com/paper/2309.06969/) (2023)
- [Distributionally Robust Recourse Action](https://mlsanity.com/paper/2302.11211/) (2023)
- [Counterfactual Explanations with Probabilistic Guarantees on their Robustness to Model Change](https://mlsanity.com/paper/2408.04842/) (2024)
- [Personalized Algorithmic Recourse with Preference Elicitation](https://mlsanity.com/paper/2205.13743/) (2022)
- [Endogenous Macrodynamics in Algorithmic Recourse](https://mlsanity.com/paper/2308.08187/) (2023)
- [Time Can Invalidate Algorithmic Recourse](https://mlsanity.com/paper/2410.08007/) (2024)
- [Feature Responsiveness Scores: Model-Agnostic Explanations for Recourse](https://mlsanity.com/paper/2410.22598/) (2024)
- [The Importance of Time in Causal Algorithmic Recourse](https://mlsanity.com/paper/2306.05082/) (2023)
- [Perfect Counterfactuals in Imperfect Worlds: Modelling Noisy Implementation of Actions in Sequential Algorithmic Recourse](https://mlsanity.com/paper/2410.02273/) (2024)
- [SafeAR: Safe Algorithmic Recourse by Risk-Aware Policies](https://mlsanity.com/paper/2308.12367/) (2023)

## Built on by (most cited first, 3 of 3)

- [ElliCE: Efficient and Provably Robust Algorithmic Recourse via the Rashomon Sets](https://mlsanity.com/paper/2602.07674/) (2026)
- [From Universal to Individualized Actionability: Revisiting Personalization in Algorithmic Recourse](https://mlsanity.com/paper/2604.08030/) (2026)
- [Algorithmic Recourse Under Competition](https://mlsanity.com/paper/2609.39877/) (2026)

## Abstract (arXiv)

Algorithmic recourse seeks to provide individuals with actionable recommendations that increase their chances of receiving favorable outcomes from automated decision systems (e.g., loan approvals). While prior research has emphasized robustness to model updates, considerably less attention has been given to the temporal dynamics of recourse--particularly in competitive, resource-constrained settings where recommendations shape future applicant pools. In this work, we present a novel time-aware framework for algorithmic recourse, explicitly modeling how candidate populations adapt in response to recommendations. Additionally, we introduce a novel reinforcement learning (RL)-based recourse algorithm that captures the evolving dynamics of the environment to generate recommendations that are both feasible and valid. We design our recommendations to be durable, supporting validity over a predefined time horizon T. This durability allows individuals to confidently reapply after taking time to implement the suggested changes. Through extensive experiments in complex simulation environments, we show that our approach substantially outperforms existing baselines, offering a superior balance between feasibility and long-term validity. Together, these results underscore the importance of incorporating temporal and behavioral dynamics into the design of practical recourse systems.