# Measurement as Bricolage: Examining How Data Scientists Construct Target Variables for Predictive Modeling Tasks

arXiv: https://arxiv.org/abs/2507.02819 · HTML: https://mlsanity.com/paper/2507.02819/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 151 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Builds on (most cited first, 30 of 58)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI](https://mlsanity.com/paper/2204.01075/) (2022)
- [Aequitas: A Bias and Fairness Audit Toolkit](https://mlsanity.com/paper/1811.05577/) (2018)
- [Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences](https://mlsanity.com/paper/2404.12272/) (2024)
- [DoWhy: An End-to-End Library for Causal Inference](https://mlsanity.com/paper/2011.04216/) (2020)
- [FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning](https://mlsanity.com/paper/1904.05419/) (2019)
- [The Fallacy of AI Functionality](https://mlsanity.com/paper/2206.09511/) (2022)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [Fairlearn: Assessing and Improving Fairness of AI Systems](https://mlsanity.com/paper/2303.16626/) (2023)
- [Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation](https://mlsanity.com/paper/2205.00501/) (2022)
- [How do Data Science Workers Collaborate? Roles, Workflows, and Tools](https://mlsanity.com/paper/2001.06684/) (2020)
- [FairSight: Visual Analytics for Fairness in Decision Making](https://mlsanity.com/paper/1908.00176/) (2019)
- [Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process](https://mlsanity.com/paper/2110.10234/) (2021)
- [Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support](https://mlsanity.com/paper/2204.02310/) (2022)
- [Medical Large Language Model Benchmarks Should Prioritize Construct Validity](https://mlsanity.com/paper/2503.10694/) (2025)
- [Human-Centered Design Recommendations for LLM-as-a-Judge](https://mlsanity.com/paper/2407.03479/) (2024)
- [Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits](https://mlsanity.com/paper/2205.06922/) (2022)
- [Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning](https://mlsanity.com/paper/2302.04732/) (2023)
- [Dataset Discovery in Data Lakes](https://mlsanity.com/paper/2011.10427/) (2020)
- [Benchmarking emergency department triage prediction models with machine learning and large public electronic health records](https://mlsanity.com/paper/2111.11017/) (2021)
- [How Data Scientists Work Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question?](https://mlsanity.com/paper/1909.03486/) (2019)
- [A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms](https://mlsanity.com/paper/2206.14983/) (2022)
- [What are you optimizing for? Aligning Recommender Systems with Human Values](https://mlsanity.com/paper/2107.10939/) (2021)

## Built on by (most cited first, 1 of 1)

- [To Build or Not to Build? Factors that Lead to Non-Development or Abandonment of AI Systems](https://mlsanity.com/paper/2604.28053/) (2026)

## Abstract (arXiv)

Data scientists often formulate predictive modeling tasks involving fuzzy, hard-to-define concepts, such as the "authenticity" of student writing or the "healthcare need" of a patient. Yet the process by which data scientists translate fuzzy concepts into a concrete, proxy target variable remains poorly understood. We interview fifteen data scientists in education (N=8) and healthcare (N=7) to understand how they construct target variables for predictive modeling tasks. Our findings suggest that data scientists construct target variables through a bricolage process, in which they use creative and pragmatic approaches to make do with the limited data at hand. Data scientists attempt to satisfy five major criteria for a target variable through bricolage: validity, simplicity, predictability, portability, and resource requirements. To achieve this, data scientists adaptively apply problem (re)formulation strategies, such as swapping out one candidate target variable for another when the first fails to meet certain criteria (e.g., predictability), or composing multiple outcomes into a single target variable to capture a more holistic set of modeling objectives. Based on our findings, we present opportunities for future HCI, CSCW, and ML research to better support the art and science of target variable construction.