# The Double-Edged Sword of Behavioral Responses in Strategic Classification: Theory and User Studies

arXiv: https://arxiv.org/abs/2410.18066 · HTML: https://mlsanity.com/paper/2410.18066/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 57 references that are in the map.
- First posted on arXiv 23 Oct 2024.

## Builds on (most cited first, 30 of 30)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Actionable Recourse in Linear Classification](https://mlsanity.com/paper/1809.06514/) (2018)
- [Performative Prediction](https://mlsanity.com/paper/2002.06673/) (2020)
- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Strategic Classification](https://mlsanity.com/paper/1506.06980/) (2015)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Algorithmic Recourse: from Counterfactual Explanations to Interventions](https://mlsanity.com/paper/2002.06278/) (2020)
- [Problems with Shapley-value-based explanations as feature importance measures](https://mlsanity.com/paper/2002.11097/) (2020)
- [Strategic Classification from Revealed Preferences](https://mlsanity.com/paper/1710.07887/) (2017)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- ["How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations](https://mlsanity.com/paper/1911.06473/) (2019)
- [Explanations Can Reduce Overreliance on AI Systems During Decision-Making](https://mlsanity.com/paper/2212.06823/) (2022)
- [The Social Cost of Strategic Classification](https://mlsanity.com/paper/1808.08460/) (2018)
- [The Disparate Effects of Strategic Manipulation](https://mlsanity.com/paper/1808.08646/) (2018)
- [Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs](https://mlsanity.com/paper/2004.11440/) (2020)
- [Strategic Classification in the Dark](https://mlsanity.com/paper/2102.11592/) (2021)
- [Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making](https://mlsanity.com/paper/2010.07938/) (2020)
- [Large Language Models Assume People are More Rational than We Really are](https://mlsanity.com/paper/2406.17055/) (2024)
- [The Strategic Perceptron](https://mlsanity.com/paper/2008.01710/) (2020)
- [Information Discrepancy in Strategic Learning](https://mlsanity.com/paper/2103.01028/) (2021)
- [The Disparate Equilibria of Algorithmic Decision Making when Individuals Invest Rationally](https://mlsanity.com/paper/1910.04123/) (2019)
- [Calibrated Stackelberg Games: Learning Optimal Commitments Against Calibrated Agents](https://mlsanity.com/paper/2306.02704/) (2023)
- [Bayesian Persuasion for Algorithmic Recourse](https://mlsanity.com/paper/2112.06283/) (2021)
- [Capturing the Complexity of Human Strategic Decision-Making with Machine Learning](https://mlsanity.com/paper/2408.07865/) (2024)
- [Do Users Benefit From Interpretable Vision? A User Study, Baseline, And Dataset](https://mlsanity.com/paper/2204.11642/) (2022)
- [Bayesian Strategic Classification](https://mlsanity.com/paper/2402.08758/) (2024)
- [Persuading a Behavioral Agent: Approximately Best Responding and Learning](https://mlsanity.com/paper/2302.03719/) (2023)
- [On Modeling Human Perceptions of Allocation Policies with Uncertain Outcomes](https://mlsanity.com/paper/2103.05827/) (2021)

## Built on by (most cited first, 9 of 9)

- [Incentivizing Desirable Effort Profiles in Strategic Classification: The Role of Causality and Uncertainty](https://mlsanity.com/paper/2502.06749/) (2025)
- [How Strategic Agents Respond: Comparing Analytical Models with LLM-Generated Responses in Strategic Classification](https://mlsanity.com/paper/2501.16355/) (2025)
- [Incentive-Aware Machine Learning; Robustness, Fairness, Improvement & Causality](https://mlsanity.com/paper/2505.05211/) (2025)
- [Anticipating Gaming to Incentivize Improvement: Guiding Agents in (Fair) Strategic Classification](https://mlsanity.com/paper/2505.05594/) (2025)
- [The Disparate Effects of Partial Information in Bayesian Strategic Learning](https://mlsanity.com/paper/2506.00627/) (2025)
- [Beyond Rational Illusion: Behaviorally Realistic Strategic Classification](https://mlsanity.com/paper/2605.19674/) (2026)
- [Off-Policy Evaluation with Strategic Agents via Local Disclosure](https://mlsanity.com/paper/2606.07308/) (2026)
- [Robust Strategic Classification under Decision-Dependent Cost Uncertainty](https://mlsanity.com/paper/2606.30136/) (2026)
- [Strategic Classification Has a Missing Lever: Audit Risk](https://mlsanity.com/paper/2609.22534/) (2026)

## Abstract (arXiv)

When humans are subject to an algorithmic decision system, they can strategically adjust their behavior accordingly (``game'' the system). While a growing line of literature on strategic classification has used game-theoretic modeling to understand and mitigate such gaming, these existing works consider standard models of fully rational agents. In this paper, we propose a strategic classification model that considers behavioral biases in human responses to algorithms. We show how misperceptions of a classifier (specifically, of its feature weights) can lead to different types of discrepancies between biased and rational agents' responses, and identify when behavioral agents over- or under-invest in different features. We also show that strategic agents with behavioral biases can benefit or (perhaps, unexpectedly) harm the firm compared to fully rational strategic agents. We complement our analytical results with user studies, which support our hypothesis of behavioral biases in human responses to the algorithm. Together, our findings highlight the need to account for human (cognitive) biases when designing AI systems, and providing explanations of them, to strategic human in the loop.