mlsanityData to 14 Oct 2026

Segment-driven Structural Induction and Semantic Alignment for Heterogeneous Tabular Representation

arXiv 2606.01890 ↗First posted 1 Jun 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Real-world domains often contain heterogeneous tables whose headers vary while their underlying attribute semantics are shared, making it difficult to induce domain-specialized semantics from table-local evidence alone. Existing encoders model parts of this problem, but often underuse column-level value distributions and apply uniform objectives across attributes with different semantic roles. We propose NAVI, a segment-centric pretraining framework that treats each header-value pair as the unit for aggregating schema-level structural evidence and column-level distributional evidence. We realize this design through Masked Segment Modeling and Entropy-driven Segment Alignment, which jointly enforce structured header-value coupling and semantic alignment across stable and instance-specific attributes. Experiments on heterogeneous in-domain tables show improved reconstruction, semantic consistency, and downstream utility across evaluation settings overall.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

26 of 36 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2016Layer Normalizationcited by 5,669
  5. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  6. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  7. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  8. 2021Revisiting Deep Learning Models for Tabular Datacited by 740
  9. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  10. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 434
  11. 2020TabTransformer: Tabular Data Modeling Using Contextual Embeddingscited by 409
  12. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 398
14 more
  1. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 377
  2. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 319
  3. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 235
  4. 2023Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Spacecited by 186
  5. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  6. 2023TableLlama: Towards Open Large Generalist Models for Tablescited by 137
  7. 2020TUTA: Tree-based Transformers for Generally Structured Table Pre-trainingcited by 100
  8. 2022STaSy: Score-based Tabular data Synthesiscited by 86
  9. 2022Diffusion models for missing value imputation in tabular datacited by 77
  10. 2024Making Pre-trained Language Models Great on Tabular Predictioncited by 57
  11. 2022UniCLIP: Unified Framework for Contrastive Language-Image Pre-trainingcited by 54
  12. 2023Towards Cross-Table Masked Pretraining for Web Data Miningcited by 21
  13. 2023Continuous Diffusion for Mixed-Type Tabular Datacited by 17
  14. 2025Multi-level Diagnosis and Evaluation for Robust Tabular Feature Engineering with Large Language Modelscited by 1

Built on by

Not cited yet.