mlsanityData to 14 Oct 2026

Beyond the Name: Demographic Leakage in De-Identified R'esum'es and Evaluation Artifacts in LLM Bias Audits

arXiv 2609.16501 ↗First posted 15 Sep 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

De-identified r'esum'e screening assumes that redacting explicit fields prevents ethnocultural inference; however, recent audits attribute residual leakage to declared languages. We investigate whether eliminating language fields resolves this leakage across nine open-weight models and 620 counterfactual r'esum'es. By holding language attributes strictly identical, we isolate unstructured prose across five ethnocultural conditions and three cue-salience tiers. Target-group recovery averages 0.757 overall and saturates at 1.000 under high salience, demonstrating that non-language prose sustains demographic inference. Crucially, models diverge only under faint cues (0.086-0.690), establishing salience as an essential evaluation axis. Furthermore, pairwise LLM-as-a-judge outcomes are highly sensitive to evaluation design: forbidding ties yields an apparent selection-rate ratio of 0.39 alongside strong position and content effects, whereas permitting ties produces near-universal ties for most models (≥94%). Downstream scoring shows only very small between-condition differences, highlighting the need to distinguish demographic signals recoverable from r'esum'e content from effects introduced by the evaluation protocol.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 16 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

7 of 16 references.

References · 7

  1. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
  2. 2025NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelcited by 108
  3. 2024Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrievalcited by 51
  4. 2024"You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendationscited by 33
  5. 2025Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Educationcited by 14
  6. 2026Small Changes, Big Impact: Demographic Bias in LLM-Based Hiring Through Subtle Sociocultural Markers in Anonymised Resumescited by 4
  7. 2026Can LLMs Hire Fairly? Racial Bias in Resume Screeningcited by 4

Built on by

Not cited yet.