# Understanding Endogenous Data Drift in Adaptive Models with Recourse-Seeking Users

arXiv: https://arxiv.org/abs/2503.09658 · HTML: https://mlsanity.com/paper/2503.09658/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 60 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Strategic Classification](https://mlsanity.com/paper/1506.06980/) (2015)
- [FACE: Feasible and Actionable Counterfactual Explanations](https://mlsanity.com/paper/1909.09369/) (2019)
- [Fairness in Machine Learning](https://mlsanity.com/paper/2012.15816/) (2020)
- [Binary Neural Networks: A Survey](https://mlsanity.com/paper/2004.03333/) (2020)
- [Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review](https://mlsanity.com/paper/2010.10596/) (2020)
- [A survey of algorithmic recourse: definitions, formulations, solutions, and prospects](https://mlsanity.com/paper/2010.04050/) (2020)
- [Towards Robust and Reliable Algorithmic Recourse](https://mlsanity.com/paper/2102.13620/) (2021)
- [Strategic Classification Made Practical](https://mlsanity.com/paper/2103.01826/) (2021)
- [Equalizing Recourse across Groups](https://mlsanity.com/paper/1909.03166/) (2019)
- [Interpretability via Model Extraction](https://mlsanity.com/paper/1706.09773/) (2017)
- [Algorithmic Recourse in the Wild: Understanding the Impact of Data and Model Shifts](https://mlsanity.com/paper/2012.11788/) (2020)
- [Deep Learning based Large Scale Visual Recommendation and Search for E-Commerce](https://mlsanity.com/paper/1703.02344/) (2017)
- [Setting the Right Expectations: Algorithmic Recourse Over Time](https://mlsanity.com/paper/2309.06969/) (2023)
- [RoCourseNet: Distributionally Robust Training of a Prediction Aware Recourse Model](https://mlsanity.com/paper/2206.00700/) (2022)
- [An ASP-Based Approach to Counterfactual Explanations for Classification](https://mlsanity.com/paper/2004.13237/) (2020)
- [Low-Cost Algorithmic Recourse for Users With Uncertain Cost Functions](https://mlsanity.com/paper/2111.01235/) (2021)
- [Fairness in Algorithmic Recourse Through the Lens of Substantive Equality of Opportunity](https://mlsanity.com/paper/2401.16088/) (2024)
- [The Risks of Recourse in Binary Classification](https://mlsanity.com/paper/2306.00497/) (2023)
- [Gradient-based Counterfactual Explanations using Tractable Probabilistic Models](https://mlsanity.com/paper/2205.07774/) (2022)

## Built on by (most cited first, 1 of 1)

- [Algorithmic Recourse Under Competition](https://mlsanity.com/paper/2609.39877/) (2026)

## Abstract (arXiv)

Deep learning models are widely used in decision-making and recommendation systems, where they typically rely on the assumption of a static data distribution between training and deployment. However, real-world deployment environments often violate this assumption. Users who receive negative outcomes may adapt their features to meet model criteria, i.e., recourse action. These adaptive behaviors create shifts in the data distribution and when models are retrained on this shifted data, a feedback loop emerges: user behavior influences the model, and the updated model in turn reshapes future user behavior. Despite its importance, this bidirectional interaction between users and models has received limited attention. In this work, we develop a general framework to model user strategic behaviors and their interactions with decision-making systems under resource constraints and competitive dynamics. Both the theoretical and empirical analyses show that user recourse behavior tends to push logistic and MLP models toward increasingly higher decision standards, resulting in higher recourse costs and less reliable recourse actions over time. To mitigate these challenges, we propose two methods--Fair-top-k and Dynamic Continual Learning (DCL)--which significantly reduce recourse cost and improve model robustness. Our findings draw connections to economic theories, highlighting how algorithmic decision-making can unintentionally reinforce a higher standard and generate endogenous barriers to entry.