Beyond Outcome Gaps: Process-Aware Fairness Diagnosis for LLM-based Multi-Agent Decision Systems
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
LLM-based multi-agent systems (MAS) are increasingly considered for high-stakes decision-making, yet outcome-based fairness audits can miss where risks arise within the decision trajectory. We present SCOPED-Hiring, a process-aware fairness diagnosis pipeline for LLM-based hiring MAS. SCOPED-Hiring constructs controlled resume variants, runs role-based hiring committees, logs over 311K structured decision trajectories, and converts trajectory fields into quantitative fairness signals organized by six diagnostic lenses: final outcome, counterfactual, process, pathway, dynamic, and design effects. SCOPED-Hiring reveals that balanced final hire rates can mask hidden trajectory unfairness in multi-agent decision trajectories: career gaps trigger suspicion, proxy cues shape qualification judgments, and identity cues lead to unequal investigation. Targeted repair guided by these diagnoses reduces total layered burden by 72.3% while shifting the hire rate by only 1.86 pp, showing that process diagnosis can guide effective repair. Project Page: https://scoped-hiring-project-page.vercel.app/
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 19 of its 48 references that are in the map. First posted on arXiv 2 Sep 2026.
Builds on
19 of 48 references.
Closest ancestors · 2
Same direction, newest first
- 2025Interactional Fairness in LLM Multi-Agent Systems: An Evaluation Framework17 May
- 2024Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions3 Oct
Other references · 17
- 2016Equality of Opportunity in Supervised Learningcited by 2,000
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023ChatDev: Communicative Agents for Software Developmentcited by 893
- 2017Counterfactual Fairnesscited by 802
- 2018Delayed Impact of Fair Machine Learningcited by 183
- 2024TradingAgents: Multi-Agents LLM Financial Trading Frameworkcited by 182
- 2019Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practicescited by 170
- 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agentscited by 169
- 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
- 2024Thinking Fair and Slow: On the Efficacy of Structured Prompts for Debiasing Language Modelscited by 44
- 2025Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agentscited by 39
- 2018Downstream Effects of Affirmative Actioncited by 34
5 more
- 2020Individual Fairness in Pipelinescited by 22
- 2017Fair Pipelinescited by 21
- 2025AI Hiring with LLMs: A Context-Aware and Explainable Multi-Agent Framework for Resume Screeningcited by 19
- 2025Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Educationcited by 13
- 2019The Price of Local Fairness in Multistage Selectioncited by 8
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025FairCoder: Probing LLM Bias in High-Stakes Decision Making via Coding Tasks
- 2026When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines
- 2026AgentFairBench: Do LLM Agents Discriminate When They Act?
- 2026Insights Generator: Systematic Corpus-Level Trace Diagnostics for LLM Agents
- 2026Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
- 2026Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation
- 2026Counterfactual Fairness Evaluation of LLM-Based Contact Center Agent Quality Assurance System
- 2026AURA-Eval: Evaluation Framework for Acting Under Risk Awareness in LLM Agent Trajectories