# Strategic Evaluation: Subjects, Evaluators, and Society

arXiv: https://arxiv.org/abs/2310.03655 · HTML: https://mlsanity.com/paper/2310.03655/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 57 references that are in the map.
- First posted on arXiv 5 Oct 2023.

## Builds on (most cited first, 17 of 17)

- [Strategic Classification](https://mlsanity.com/paper/1506.06980/) (2015)
- [Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices](https://mlsanity.com/paper/1906.09208/) (2019)
- [The Social Cost of Strategic Classification](https://mlsanity.com/paper/1808.08460/) (2018)
- [Strategic Classification is Causal Modeling in Disguise](https://mlsanity.com/paper/1910.10362/) (2019)
- [How Do Classifiers Induce Agents To Invest Effort Strategically?](https://mlsanity.com/paper/1807.05307/) (2018)
- [The Disparate Effects of Strategic Manipulation](https://mlsanity.com/paper/1808.08646/) (2018)
- [Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance](https://mlsanity.com/paper/2206.04737/) (2022)
- [Causal Strategic Linear Regression](https://mlsanity.com/paper/2002.10066/) (2020)
- [Who Leads and Who Follows in Strategic Classification?](https://mlsanity.com/paper/2106.12529/) (2021)
- [Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem](https://mlsanity.com/paper/2310.02521/) (2023)
- [A Short-term Intervention for Long-term Fairness in the Labor Market](https://mlsanity.com/paper/1712.00064/) (2017)
- [Stateful Strategic Regression](https://mlsanity.com/paper/2106.03827/) (2021)
- [Bridging Machine Learning and Mechanism Design towards Algorithmic Fairness](https://mlsanity.com/paper/2010.05434/) (2020)
- [Strategyproof Linear Regression in High Dimensions](https://mlsanity.com/paper/1805.10693/) (2018)
- [Algorithmic Auditing and Social Justice: Lessons from the History of Audit Studies](https://mlsanity.com/paper/2109.06974/) (2021)
- [Strategic Ranking](https://mlsanity.com/paper/2109.08240/) (2021)
- [Mechanism Design for Social Good](https://mlsanity.com/paper/1810.09832/) (2018)

## Abstract (arXiv)

A broad current application of algorithms is in formal and quantitative measures of murky concepts -- like merit -- to make decisions. When people strategically respond to these sorts of evaluations in order to gain favorable decision outcomes, their behavior can be subjected to moral judgments. They may be described as 'gaming the system' or 'cheating,' or (in other cases) investing 'honest effort' or 'improving.' Machine learning literature on strategic behavior has tried to describe these dynamics by emphasizing the efforts expended by decision subjects hoping to obtain a more favorable assessment -- some works offer ways to preempt or prevent such manipulations, some differentiate 'gaming' from 'improvement' behavior, while others aim to measure the effort burden or disparate effects of classification systems. We begin from a different starting point: that the design of an evaluation itself can be understood as furthering goals held by the evaluator which may be misaligned with broader societal goals. To develop the idea that evaluation represents a strategic interaction in which both the evaluator and the subject of their evaluation are operating out of self-interest, we put forward a model that represents the process of evaluation using three interacting agents: a decision subject, an evaluator, and society, representing a bundle of values and oversight mechanisms. We highlight our model's applicability to a number of social systems where one or two players strategically undermine the others' interests to advance their own. Treating evaluators as themselves strategic allows us to re-cast the scrutiny directed at decision subjects, towards the incentives that underpin institutional designs of evaluations. The moral standing of strategic behaviors often depend on the moral standing of the evaluations and incentives that provoke such behaviors.