# Beyond Outcome Gaps: Process-Aware Fairness Diagnosis for LLM-based Multi-Agent Decision Systems

arXiv: https://arxiv.org/abs/2609.02092 · HTML: https://mlsanity.com/paper/2609.02092/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 19 of its 48 references that are in the map.
- First posted on arXiv 2 Sep 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 19 of 19)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [TradingAgents: Multi-Agents LLM Financial Trading Framework](https://mlsanity.com/paper/2412.20138/) (2024)
- [Delayed Impact of Fair Machine Learning](https://mlsanity.com/paper/1803.04383/) (2018)
- [MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents](https://mlsanity.com/paper/2503.01935/) (2025)
- [Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices](https://mlsanity.com/paper/1906.09208/) (2019)
- [Evaluating and Mitigating Discrimination in Language Model Decisions](https://mlsanity.com/paper/2312.03689/) (2023)
- [Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions](https://mlsanity.com/paper/2410.02584/) (2024)
- [Thinking Fair and Slow: On the Efficacy of Structured Prompts for Debiasing Language Models](https://mlsanity.com/paper/2405.10431/) (2024)
- [Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents](https://mlsanity.com/paper/2502.20073/) (2025)
- [Downstream Effects of Affirmative Action](https://mlsanity.com/paper/1808.09004/) (2018)
- [Individual Fairness in Pipelines](https://mlsanity.com/paper/2004.05167/) (2020)
- [Fair Pipelines](https://mlsanity.com/paper/1707.00391/) (2017)
- [AI Hiring with LLMs: A Context-Aware and Explainable Multi-Agent Framework for Resume Screening](https://mlsanity.com/paper/2504.02870/) (2025)
- [Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education](https://mlsanity.com/paper/2503.19182/) (2025)
- [The Price of Local Fairness in Multistage Selection](https://mlsanity.com/paper/1906.06613/) (2019)
- [Interactional Fairness in LLM Multi-Agent Systems: An Evaluation Framework](https://mlsanity.com/paper/2505.12001/) (2025)

## Abstract (arXiv)

LLM-based multi-agent systems (MAS) are increasingly considered for high-stakes decision-making, yet outcome-based fairness audits can miss where risks arise within the decision trajectory. We present SCOPED-Hiring, a process-aware fairness diagnosis pipeline for LLM-based hiring MAS. SCOPED-Hiring constructs controlled resume variants, runs role-based hiring committees, logs over 311K structured decision trajectories, and converts trajectory fields into quantitative fairness signals organized by six diagnostic lenses: final outcome, counterfactual, process, pathway, dynamic, and design effects. SCOPED-Hiring reveals that balanced final hire rates can mask hidden trajectory unfairness in multi-agent decision trajectories: career gaps trigger suspicion, proxy cues shape qualification judgments, and identity cues lead to unequal investigation. Targeted repair guided by these diagnoses reduces total layered burden by 72.3% while shifting the hire rate by only 1.86 pp, showing that process diagnosis can guide effective repair. Project Page: https://scoped-hiring-project-page.vercel.app/