mlsanityData to 7 Oct 2026

FairJob: A Real-World Dataset for Fairness in Online Systems

arXiv 2407.03059 ↗First posted 3 Jul 2024cs.LG cs.AI cs.CYcited by 6

Not on the map. Why?

Abstract (arXiv)

We introduce a fairness-aware dataset for job recommendations in advertising, designed to foster research in algorithmic fairness within real-world scenarios. It was collected and prepared to comply with privacy standards and business confidentiality. An additional challenge is the lack of access to protected user attributes such as gender, for which we propose a solution to obtain a proxy estimate. Despite being anonymized and including a proxy for a sensitive attribute, our dataset preserves predictive power and maintains a realistic and challenging benchmark. This dataset addresses a significant gap in the availability of fairness-focused resources for high-impact domains like advertising -- the actual impact being having access or not to precious employment opportunities, where balancing fairness and utility is a common industrial challenge. We also explore various stages in the advertising process where unfairness can occur and introduce a method to compute a fair utility metric for the job recommendations in online systems case from a biased dataset. Experimental evaluations of bias mitigation techniques on the released dataset demonstrate potential improvements in fairness and the associated trade-offs with utility. The dataset is hosted at https://huggingface.co/datasets/criteo/FairJob. Source code for the experiments is hosted at https://github.com/criteo-research/FairJob-dataset/.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 50 of its 82 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

50 of 82 references.

References · 50

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  4. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  5. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  6. 2017Counterfactual Fairnesscited by 802
  7. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  8. 2021Tabular Data: Deep Learning is Not All You Needcited by 530
  9. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  10. 2016Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatmentcited by 368
  11. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
  12. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
38 more
  1. 2020Fairness without Demographics through Adversarially Reweighted Learningcited by 251
  2. 2020Bias and Debias in Recommender System: A Survey and Future Directionscited by 239
  3. 2018Fairness of Exposure in Rankingscited by 189
  4. 2022On Embeddings for Numerical Features in Tabular Deep Learningcited by 185
  5. 2021A survey on datasets for fairness-aware machine learningcited by 157
  6. 2018Classification with Fairness Constraints: A Meta-Algorithm with Provable Guaranteescited by 146
  7. 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
  8. 2019On the Fairness of Disentangled Representationscited by 144
  9. 2019Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Searchcited by 137
  10. 2022Bias Mitigation for Machine Learning Classifiers: A Comprehensive Surveycited by 125
  11. 2024Why Tabular Foundation Models Should Be a Research Prioritycited by 103
  12. 2018Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobservedcited by 94
  13. 2019Policy Learning for Fairness in Rankingcited by 74
  14. 2020Controlling Fairness and Bias in Dynamic Learning-to-Rankcited by 67
  15. 2023FACET: Fairness in Computer Vision Evaluation Benchmarkcited by 58
  16. 2020"What We Can't Measure, We Can't Understand": Challenges to Demographic Data Procurement in the Pursuit of Fairnesscited by 56
  17. 2020Open Benchmarking for Click-Through Rate Predictioncited by 53
  18. 2023VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolutioncited by 52
  19. 2019Equalized odds postprocessing under imperfect group informationcited by 48
  20. 2016Display Advertising with Real-Time Bidding (RTB) and Behavioural Targetingcited by 42
  21. 2018Blind Justice: Fairness with Encrypted Sensitive Attributescited by 41
  22. 2022Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML Evaluationcited by 39
  23. 2023FinalMLP: An Enhanced Two-Stream MLP Model for CTR Predictioncited by 37
  24. 2021The AI Index 2021 Annual Reportcited by 34
  25. 2023Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Datacited by 31
  26. 2018Proxy Fairnesscited by 29
  27. 2023Consensus and Subjectivity of Skin Tone Annotation for ML Fairnesscited by 22
  28. 2023Ethical Considerations for Responsible Data Curationcited by 17
  29. 2024Balanced Mixed-Type Tabular Data Synthesis with Diffusion Modelscited by 12
  30. 2023Improving Fairness and Robustness in End-to-End Speech Recognition through unsupervised clusteringcited by 11
  31. 2019A Distributed Fair Machine Learning Framework with Private Demographic Data Protectioncited by 8
  32. 2024Privacy-preserving data release leveraging optimal transport and particle gradient descentcited by 8
  33. 2023Towards Fairness in Personalized Ads Using Impression Variance Aware Reinforcement Learningcited by 6
  34. 2022Fast online ranking with fairness of exposurecited by 5
  35. 2023A benchmark of categorical encoders for binary classificationcited by 5
  36. 2022Towards Trustworthy AI-Empowered Real-Time Bidding for Online Advertisement Auctioningcited by 4
  37. 2023WCLD: Curated Large Dataset of Criminal Cases from Wisconsin Circuit Courtscited by 3
  38. 2023Trading-off price for data quality to achieve fair online allocationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Benchmarking Fairness-aware Graph Neural Networks in Knowledge Graphscited by 2
  2. 2026AIFS-SUBS: Extending Data-Driven Forecasting to Sub-Seasonal Timescalescited by 1
  3. 2025A Critical Review of Predominant Bias in Neural Networkscited by 0
  4. 2025Optimal Transport Learning: Balancing Value Optimization and Fairness in Individualized Treatment Rulescited by 0
  5. 2025HALF: Harm-Aware LLM Fairness Evaluation Aligned with Deploymentcited by 0
  6. 2026Parameterized Fair Resource Allocation under Diversity Constraintscited by 0