# Who Leads? Comparing Human-Centric and Model-Centric Strategies for Defining ML Target Variables

arXiv: https://arxiv.org/abs/2510.25974 · HTML: https://mlsanity.com/paper/2510.25974/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 65 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Builds on (most cited first, 19 of 19)

- [AutoML: A Survey of the State-of-the-Art](https://mlsanity.com/paper/1908.00709/) (2019)
- [FLAML: A Fast and Lightweight AutoML Library](https://mlsanity.com/paper/1911.04706/) (2019)
- [The Fallacy of AI Functionality](https://mlsanity.com/paper/2206.09511/) (2022)
- [Automated Data Slicing for Model Validation:A Big data - AI Integration Approach](https://mlsanity.com/paper/1807.06068/) (2018)
- [How do Data Science Workers Collaborate? Roles, Workflows, and Tools](https://mlsanity.com/paper/2001.06684/) (2020)
- [AutoML to Date and Beyond: Challenges and Opportunities](https://mlsanity.com/paper/2010.10777/) (2020)
- [It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks](https://mlsanity.com/paper/2106.05498/) (2021)
- [Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process](https://mlsanity.com/paper/2110.10234/) (2021)
- [Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology](https://mlsanity.com/paper/2003.05155/) (2020)
- [Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support](https://mlsanity.com/paper/2204.02310/) (2022)
- [Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics](https://mlsanity.com/paper/2005.07572/) (2020)
- [A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms](https://mlsanity.com/paper/2206.14983/) (2022)
- [Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making](https://mlsanity.com/paper/2302.06503/) (2023)
- [The Impact of Imperfect XAI on Human-AI Decision-Making](https://mlsanity.com/paper/2307.13566/) (2023)
- [A User-based Visual Analytics Workflow for Exploratory Model Analysis](https://mlsanity.com/paper/1809.10782/) (2018)
- [Orienting, Framing, Bridging, Magic, and Counseling: How Data Scientists Navigate the Outer Loop of Client Collaborations in Industry and Academia](https://mlsanity.com/paper/2105.05849/) (2021)
- [Visus: An Interactive System for Automatic Machine Learning Model Building and Curation](https://mlsanity.com/paper/1907.02889/) (2019)
- [Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks](https://mlsanity.com/paper/2502.10526/) (2025)
- [Enabling Collaborative Data Science Development with the Ballet Framework](https://mlsanity.com/paper/2012.07816/) (2020)

## Abstract (arXiv)

Predictive modeling has the potential to enhance human decision-making. However, many predictive models fail in practice due to problematic problem formulation in cases where the prediction target is an abstract concept or construct and practitioners need to define an appropriate target variable as a proxy to operationalize the construct of interest. The choice of an appropriate proxy target variable is rarely self-evident in practice, requiring both domain knowledge and iterative data modeling. This process is inherently collaborative, involving both domain experts and data scientists. In this work, we explore how human-machine teaming can support this process by accelerating iterations while preserving human judgment. We study the impact of two human-machine teaming strategies on proxy construction: 1) relevance-first: humans leading the process by selecting relevant proxies, and 2) performance-first: machines leading the process by recommending proxies based on predictive performance. Based on a controlled user study of a proxy construction task (N = 20), we show that the performance-first strategy facilitated faster iterations and decision-making, but also biased users towards well-performing proxies that are misaligned with the application goal. Our study highlights the opportunities and risks of human-machine teaming in operationalizing machine learning target variables, yielding insights for future research to explore the opportunities and mitigate the risks.