mlsanityData to 14 Oct 2026

Towards Universal Debiasing for Language Models-based Tabular Data Generation

arXiv 2509.16475 ↗First posted 20 Sep 2025cs.LG cs.CLcited by 1

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Large language models (LLMs) have achieved promising results in tabular data generation. However, inherent historical biases in tabular datasets often cause LLMs to exacerbate fairness issues, particularly when multiple advantaged and protected features are involved. In this work, we introduce a universal debiasing framework that minimizes group-level dependencies by simultaneously reducing the mutual information between advantaged and protected attributes. By leveraging the autoregressive structure and analytic sampling distributions of LLM-based tabular data generators, our approach efficiently computes mutual information, reducing the need for cumbersome numerical estimations. Building on this foundation, we propose two complementary methods: a direct preference optimization (DPO)-based strategy, namely UDF-DPO, that integrates seamlessly with existing models, and a targeted debiasing technique, namely UDF-MIX, that achieves debiasing without tuning the parameters of LLMs. Extensive experiments demonstrate that our framework effectively balances fairness and utility, offering a scalable and practical solution for debiasing in high-stakes applications.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 31 of its 54 references that are in the map. First posted on arXiv 20 Sep 2025.

Builds on

31 of 54 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment29 Feb
  2. 2023A General Theoretical Paradigm to Understand Learning from Human Preferences18 Oct

Other references · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2022Generative Adversarial Networkscited by 5,627
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  9. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  10. 2019How Can We Know What Language Models Know?cited by 959
  11. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  12. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
17 more
  1. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
  2. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
  3. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
  4. 2022Language Models are Realistic Tabular Data Generatorscited by 294
  5. 2018FairGAN: Fairness-aware Generative Adversarial Networkscited by 164
  6. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  7. 2021Fair Mixup: Fairness via Interpolationcited by 109
  8. 2021DECAF: Generating Fair Synthetic Data Using Causally-Aware Generative Networkscited by 109
  9. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 95
  10. 2023Generative Table Pre-training Empowers Models for Tabular Predictioncited by 55
  11. 2023TabuLa: Harnessing Language Models for Tabular Data Synthesiscited by 42
  12. 2023A Survey on Intersectional Fairness in Machine Learning: Notions, Mitigation, and Challengescited by 41
  13. 2023TabMT: Generating tabular data with masked transformerscited by 36
  14. 2022Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimizationcited by 33
  15. 2024Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolutioncited by 19
  16. 2025Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuningcited by 19
  17. 2024Counterfactual Fairness by Combining Factual and Counterfactual Predictionscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2024Improving LLM Group Fairness on Tabular Data via In-Context Learning
  2. 2025BiasFilter: An Inference-Time Debiasing Framework for Large Language Models
  3. 2026DebiasRAG: A Tuning-Free Path to Fair Generation in Large Language Models through Retrieval-Augmented Generation
  4. 2024BiasDPO: Mitigating Bias in Language Models through Direct Preference Optimization
  5. 2023Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classifications
  6. 2025Exploring the Heterogeneity of Tabular Data: A Diversity-aware Data Generator via LLMs
  7. 2025In-Context Bias Propagation in LLM-Based Tabular Data Generation
  8. 2026Mutual Debiasing via Dual-Seed Comparison for Probabilistic Sampling in Large Language Models