# Subverting machines, fluctuating identities: Re-learning human categorization

arXiv: https://arxiv.org/abs/2205.13740 · HTML: https://mlsanity.com/paper/2205.13740/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 45 references that are in the map.
- First posted on arXiv 27 May 2022.

## Builds on (most cited first, 9 of 9)

- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Towards a Critical Race Methodology in Algorithmic Fairness](https://mlsanity.com/paper/1912.03593/) (2019)
- [Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence](https://mlsanity.com/paper/2007.04068/) (2020)
- [The Use and Misuse of Counterfactuals in Ethical Machine Learning](https://mlsanity.com/paper/2102.05085/) (2021)
- [Connecting Generative Adversarial Networks and Actor-Critic Methods](https://mlsanity.com/paper/1610.01945/) (2016)
- [Bringing the People Back In: Contesting Benchmark Machine Learning Datasets](https://mlsanity.com/paper/2007.07399/) (2020)
- [Efficient Model Learning for Human-Robot Collaborative Tasks](https://mlsanity.com/paper/1405.6341/) (2014)
- [Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision](https://mlsanity.com/paper/2007.14886/) (2020)
- [Epistemic values in feature importance methods: Lessons from feminist epistemology](https://mlsanity.com/paper/2101.12737/) (2021)

## Built on by (most cited first, 10 of 10)

- [Queer In AI: A Case Study in Community-Led Participatory AI](https://mlsanity.com/paper/2303.16972/) (2023)
- [How Hate Speech Varies by Target Identity: A Computational Analysis](https://mlsanity.com/paper/2210.10839/) (2022)
- [Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms](https://mlsanity.com/paper/2307.10223/) (2023)
- [Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor](https://mlsanity.com/paper/2506.14652/) (2025)
- [Evaluating Model Bias Requires Characterizing its Mistakes](https://mlsanity.com/paper/2407.10633/) (2024)
- [Unraveling the Interconnected Axes of Heterogeneity in Machine Learning for Democratic and Inclusive Advancements](https://mlsanity.com/paper/2306.10043/) (2023)
- [Fair Enough? A map of the current limitations of the requirements to have fair algorithms](https://mlsanity.com/paper/2311.12435/) (2023)
- [(Unfair) Norms in Fairness Research: A Meta-Analysis](https://mlsanity.com/paper/2407.16895/) (2024)
- [What's the Problem, Linda? The Conjunction Fallacy as a Fairness Problem](https://mlsanity.com/paper/2305.09535/) (2023)
- [Representation Fidelity:Auditing Algorithmic Decisions About Humans Using Self-Descriptions](https://mlsanity.com/paper/2603.05136/) (2026)

## Abstract (arXiv)

Most machine learning systems that interact with humans construct some notion of a person's "identity," yet the default paradigm in AI research envisions identity with essential attributes that are discrete and static. In stark contrast, strands of thought within critical theory present a conception of identity as malleable and constructed entirely through interaction; a doing rather than a being. In this work, we distill some of these ideas for machine learning practitioners and introduce a theory of identity as autopoiesis, circular processes of formation and function. We argue that the default paradigm of identity used by the field immobilizes existing identity categories and the power differentials that counicodex2010occur, due to the absence of iterative feedback to our models. This includes a critique of emergent AI fairness practices that continue to impose the default paradigm. Finally, we apply our theory to sketch approaches to autopoietic identity through multilevel optimization and relational learning. While these ideas raise many open questions, we imagine the possibilities of machines that are capable of expressing human identity as a relationship perpetually in flux.