# From If-Statements to ML Pipelines: Revisiting Bias in Code-Generation

arXiv: https://arxiv.org/abs/2604.21716 · HTML: https://mlsanity.com/paper/2604.21716/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 23 Apr 2026.

## Builds on (most cited first, 14 of 14)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation](https://mlsanity.com/paper/2310.03302/) (2023)
- [CodeGemma: Open Code Models Based on Gemma](https://mlsanity.com/paper/2406.11409/) (2024)
- [Bias Testing and Mitigation in LLM-based Code Generation](https://mlsanity.com/paper/2309.14345/) (2023)
- [Uncovering and Quantifying Social Biases in Code Generation](https://mlsanity.com/paper/2305.15377/) (2023)
- [Bias Unveiled: Investigating Social Bias in LLM-Generated Code](https://mlsanity.com/paper/2411.10351/) (2024)
- [A Study of LLMs' Preferences for Libraries and Programming Languages](https://mlsanity.com/paper/2503.17181/) (2025)
- [Exploring Multi-Lingual Bias of Large Code Models in Code Generation](https://mlsanity.com/paper/2404.19368/) (2024)
- [FairCoder: Probing LLM Bias in High-Stakes Decision Making via Coding Tasks](https://mlsanity.com/paper/2501.05396/) (2025)
- [Mitigating Gender Bias in Code Large Language Models via Model Editing](https://mlsanity.com/paper/2410.07820/) (2024)

## Abstract (arXiv)

Prior work evaluates code generation bias primarily through simple conditional statements, which represent only a narrow slice of real-world programming and reveal solely overt, explicitly encoded bias. We demonstrate that this approach dramatically underestimates bias in practice by examining a more realistic task: generating machine learning (ML) pipelines. Testing both code-specialized and general-instruction large language models, we find that generated pipelines exhibit significant bias during feature selection. Sensitive attributes appear in 87.7% of cases on average, despite models demonstrably excluding irrelevant features (e.g., including "race" while dropping "favorite color" for credit scoring). This bias is substantially more prevalent than that captured by conditional statements, where sensitive attributes appear in only 59.2% of cases. These findings are robust across prompt mitigation strategies, varying numbers of attributes, and different pipeline difficulty levels. Our results challenge simple conditionals as valid proxies for bias evaluation and suggest current benchmarks underestimate bias risk in practical deployments.