mlsanityData to 7 Oct 2026

Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML Evaluation

arXiv 2211.13358 ↗First posted 24 Nov 2022cs.LGcited by 39

Not on the map. Why?

Abstract (arXiv)

Evaluating new techniques on realistic datasets plays a crucial role in the development of ML research and its broader adoption by practitioners. In recent years, there has been a significant increase of publicly available unstructured data resources for computer vision and NLP tasks. However, tabular data -- which is prevalent in many high-stakes domains -- has been lagging behind. To bridge this gap, we present Bank Account Fraud (BAF), the first publicly available privacy-preserving, large-scale, realistic suite of tabular datasets. The suite was generated by applying state-of-the-art tabular data generation techniques on an anonymized,real-world bank account opening fraud detection dataset. This setting carries a set of challenges that are commonplace in real-world applications, including temporal dynamics and significant class imbalance. Additionally, to allow practitioners to stress test both performance and fairness of ML methods, each dataset variant of BAF contains specific types of data bias. With this resource, we aim to provide the research community with a more realistic, complete, and robust test bed to evaluate novel and existing methods.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 28 of its 55 references that are in the map. First posted on arXiv 24 Nov 2022.

Builds on

28 of 55 references.

References · 28

  1. 2018Datasheets for Datasetscited by 1,371
  2. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  3. 2019Modeling Tabular data using Conditional GANcited by 695
  4. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 443
  5. 2017Algorithmic decision making and the cost of fairnesscited by 402
  6. 2020Performative Predictioncited by 318
  7. 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 259
  8. 2018Differentially Private Generative Adversarial Networkcited by 235
  9. 2018A comparative study of fairness-enhancing interventions in machine learningcited by 208
  10. 2020Fairness in Machine Learningcited by 194
  11. 2021A survey on datasets for fairness-aware machine learningcited by 157
  12. 2021CTAB-GAN: Effective Table Data Synthesizingcited by 156
16 more
  1. 2021Pros and Cons of GAN Evaluation Measures: New Developmentscited by 121
  2. 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 100
  3. 2021Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Researchcited by 100
  4. 2021Benchmarking Differentially Private Synthetic Data Generation Algorithmscited by 80
  5. 2021It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarkscited by 61
  6. 2022Algorithmic Fairness Datasets: the Story so Farcited by 51
  7. 2020Differentially Private Synthetic Data: Applied Evaluations and Enhancementscited by 36
  8. 2021The AI Index 2021 Annual Reportcited by 34
  9. 2018Differentially Private Data Generative Modelscited by 33
  10. 2019Value-laden Disciplinary Shifts in Machine Learningcited by 25
  11. 2019ProPublica's COMPAS Data Revisitedcited by 24
  12. 2021A Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communicationcited by 18
  13. 2021An Empirical Comparison of Bias Reduction Methods on Real-World Problems in High-Stakes Policy Settingscited by 8
  14. 2022Understanding Unfairness in Fraud Detection through Model and Data Bias Interactionscited by 7
  15. 2013Utilizing Noise Addition for Data Privacy, an Overviewcited by 6
  16. 2022Prisoners of Their Own Devices: How Models Induce Data Bias in Performative Predictioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 15; 2025: 11; 2026: 8.

39 papers.

Sort:

By area:

  1. 2025Representation Learning for Tabular Data: A Comprehensive Surveycited by 36
  2. 2023Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Datacited by 31
  3. 2024A Survey on Evaluation of Out-of-Distribution Generalizationcited by 20
  4. 2023Transparency and Privacy: The Role of Explainable AI and Federated Learning in Financial Fraud Detectioncited by 13
  5. 2024Cost-Sensitive Learning to Defer to Multiple Experts with Workload Constraintscited by 12
  6. 2024Balanced Mixed-Type Tabular Data Synthesis with Diffusion Modelscited by 12
  7. 2024Causality for Tabular Data Synthesis: A High-Order Structure Causal Benchmark Frameworkcited by 7
  8. 2024FairJob: A Real-World Dataset for Fairness in Online Systemscited by 6
  9. 2025CLIMB: Class-imbalanced Learning Benchmark on Tabular Datacited by 6
  10. 2023Fairness-Aware Data Valuation for Supervised Learningcited by 4
  11. 2024Aequitas Flow: Streamlining Fair ML Experimentationcited by 4
  12. 2026FAMOSE: A ReAct Approach to Automated Feature Discoverycited by 4
27 more
  1. 2023FiFAR: A Fraud Detection Dataset for Learning to Defercited by 3
  2. 2024Is it Still Fair? A Comparative Evaluation of Fairness Algorithms through the Lens of Covariate Driftcited by 3
  3. 2024Kolmogorov Arnold Networks in Fraud Detection: Bridging the Gap Between Theory and Practicecited by 2
  4. 2025Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attackscited by 2
  5. 2023Transferable Adversarial Robustness for Categorical Data via Universal Robust Embeddingscited by 1
  6. 2024Decoupling Decision-Making in Fraud Prevention through Classifier Calibration for Business Logic Actioncited by 1
  7. 2024A Note on Bias to Completecited by 1
  8. 2024RIFF: Inducing Rules for Fraud Detection from Decision Treescited by 1
  9. 2024Fair-OBNC: Correcting Label Noise for Fairer Datasetscited by 1
  10. 2025PnPXAI: A Universal XAI Framework Providing Automatic Explanations Across Diverse Modalities and Modelscited by 1
  11. 2025Fair for a few: Improving Fairness in Doubly Imbalanced Datasetscited by 1
  12. 2025Evaluating Transfer Learning Methods on Real-World Data Streams: A Case Study in Financial Fraud Detectioncited by 1
  13. 2024On Ranking-based Tests of Independencecited by 0
  14. 2024"Show Me What's Wrong!": Combining Charts and Text to Guide Data Analysiscited by 0
  15. 2024Undermining Image and Text Classification Algorithms Using Adversarial Attackscited by 0
  16. 2025A Critical Review of Predominant Bias in Neural Networkscited by 0
  17. 2025Data Heterogeneity Modeling for Trustworthy Machine Learningcited by 0
  18. 2025GFLC: Graph-based Fairness-aware Label Correction for Fair Classificationcited by 0
  19. 2025LARP: Learner-Agnostic Robust Data Prefilteringcited by 0
  20. 2025Reinforcement-Guided Hyper-Heuristic Hyperparameter Optimization for Fair and Explainable Spiking Neural Network-Based Financial Fraud Detectioncited by 0
  21. 2026Adaptive Sampling and Clipping for Private Worst-Case Group Optimizationcited by 0
  22. 2026Efficient and Interpretable Transformer for Counterfactual Fairnesscited by 0
  23. 2026Imbalanced Classification under Capacity Constraintscited by 0
  24. 2026A Drift Stable Quantum Federated Learning for Intelligent Servicescited by 0
  25. 2026When Can Fraud Operations Authorize Automation? A Decision-Support Framework for Fresh Audit Evidence and Review Workloadcited by 0
  26. 2026A Stable Aggregation Method for Quantum Federated Learningcited by 0
  27. 2026Collaborative Synthetic Data for Privacy-Preserving Financial Fraud Detection Across Organizational Siloscited by 0