mlsanityData to 14 Oct 2026

Not All Features Deserve Attention: Graph-Guided Dependency Learning for Tabular Data Generation with Language Models

arXiv 2507.18504 ↗First posted 24 Jul 2025cs.CL cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have shown strong potential for tabular data generation by modeling textualized feature-value pairs. However, tabular data inherently exhibits sparse feature-level dependencies, where many feature interactions are structurally insignificant. This creates a fundamental mismatch as LLMs' self-attention mechanism inevitably distributes focus across all pairs, diluting attention on critical relationships, particularly in datasets with complex dependencies or semantically ambiguous features. To address this limitation, we propose GraDe (Graph-Guided Dependency Learning), a novel method that explicitly integrates sparse dependency graphs into LLMs' attention mechanism. GraDe employs a lightweight dynamic graph learning module guided by externally extracted functional dependencies, prioritizing key feature interactions while suppressing irrelevant ones. Our experiments across diverse real-world datasets demonstrate that GraDe outperforms existing LLM-based approaches by up to 12% on complex datasets while achieving competitive results with state-of-the-art approaches in synthetic data quality. Our method is minimally intrusive yet effective, offering a practical solution for structure-aware tabular data modeling with LLMs.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 33 of its 73 references that are in the map. First posted on arXiv 24 Jul 2025.

Builds on

33 of 73 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2020Longformer: The Long-Document Transformercited by 3,413
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  5. 2020Reformer: The Efficient Transformercited by 1,668
  6. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  7. 2019Modeling Tabular data using Conditional GANcited by 699
  8. 2021Deep Neural Networks and Tabular Data: A Surveycited by 491
  9. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 449
  10. 2017Real-valued (Medical) Time Series Generation with Recurrent Conditional GANscited by 333
  11. 2022Language Models are Realistic Tabular Data Generatorscited by 294
  12. 2020Synthesizer: Rethinking Self-Attention in Transformer Modelscited by 205
21 more
  1. 2023Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Spacecited by 186
  2. 2021CTAB-GAN: Effective Table Data Synthesizingcited by 156
  3. 2021GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graphcited by 143
  4. 2019An Attentive Survey of Attention Modelscited by 124
  5. 2024Why Tabular Foundation Models Should Be a Research Prioritycited by 106
  6. 2024Large Scale Transfer Learning for Tabular Data via Language Modelingcited by 76
  7. 2022CTAB-GAN+: Enhancing Tabular Data Synthesiscited by 70
  8. 2022A General Survey on Attention Mechanisms in Deep Learningcited by 64
  9. 2025PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selectioncited by 61
  10. 2024Making Pre-trained Language Models Great on Tabular Predictioncited by 57
  11. 2023Generative Table Pre-training Empowers Models for Tabular Predictioncited by 55
  12. 2023Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimescited by 52
  13. 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 51
  14. 2023TabMT: Generating tabular data with masked transformerscited by 36
  15. 2024CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitionscited by 14
  16. 2020Trajectories, bifurcations and pseudotime in large clinical datasets: applications to myocardial infarction and diabetes datacited by 13
  17. 2023On the Usefulness of Synthetic Tabular Data Generationcited by 12
  18. 2024Why LLMs Are Bad at Synthetic Table Generation (and what to do about it)cited by 10
  19. 2023Baby's CoThought: Leveraging Large Language Models for Enhanced Reasoning in Compact Modelscited by 9
  20. 2025MultiConIR: Towards multi-condition Information Retrievalcited by 9
  21. 2025ConsistentChat: Building Skeleton-Guided Consistent Multi-Turn Dialogues for Large Language Models from Scratchcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

  1. 2025Doubling Your Data in Minutes: Ultra-fast Tabular Data Generation via LLM-Induced Dependency Graphscited by 9
  2. 2025Attributes as Textual Genes: Leveraging LLMs as Genetic Algorithm Simulators for Conditional Synthetic Data Generationcited by 5
  3. 2026Model-Agnostic Meta Learning for Class Imbalance Adaptationcited by 1
  4. 2026SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generationcited by 1
  5. 2026Active Tabular Augmentation via Policy-Guided Diffusion Inpaintingcited by 1
  6. 2025StructSynth: Dependency Graphs as Generation Plans for Low-Data Tabular Synthesis with Language Modelscited by 0
  7. 2025SeaPO: Strategic Error Amplification for Robust Preference Optimization of Large Language Modelscited by 0
  8. 2026Relational Learning as World Modelingcited by 0