mlsanityData to 18 Nov 2026

LightTAR: Light and Fast Target-Aware Representations for Multimodal Tabular Learning

arXiv 2611.17600 ↗Submitted 12 Nov 2026cs.LG cs.CL cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal tabular learning involves classification and regression on tables combining structured numerical and categorical features with unstructured text and images. These unstructured fields are commonly represented by pretrained encoder embeddings reduced with principal component analysis (PCA). This reduction prioritizes input variance but may overlook information relevant to predicting the target. We introduce LightTAR, a simple, inexpensive method that adapts dimensionality reduction to the prediction task. LightTAR augments the standard PCA representation with a few target-aware features extracted jointly from the final and an intermediate encoder layer using regularized partial least squares (PLS). We evaluate LightTAR with 21 tabular models, including tree ensembles, neural networks, and tabular foundation models. LightTAR improves average predictive performance for all models and, averaged across models, improves prediction on almost every MulTaBench dataset. These gains come with a small increase in total runtime relative to PCA representations. LightTAR integrates directly into existing encoding pipelines, improving tabular learners' predictive performance without modifying the encoder or the learner.

Key facts. No citing papers in the map yet (data through 18 Nov 2026). Builds on 34 of its 47 references that are in the map. Submitted 12 Nov 2026.

Builds on

34 of 47 references.

References · 34

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,496
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,339
  3. 2018Deep contextualized word representationscited by 2,315
  4. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,348
  5. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,097
  6. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 788
  7. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 328
  8. 2025TabArena: A Living Benchmark for Machine Learning on Tabular Datacited by 200
  9. 2026TabICLv2: A better, faster, scalable, and open tabular foundation modelcited by 184
  10. 2024TabDPT: Scaling Tabular Foundation Models on Real Datacited by 130
  11. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 129
  12. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 114
22 more
  1. 2024Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Datacited by 101
  2. 2022Head2Toe: Utilizing Intermediate Representations for Better Transfer Learningcited by 84
  3. 2024Large Scale Transfer Learning for Tabular Data via Language Modelingcited by 84
  4. 2025ConTextTab: A Semantics-Aware Tabular In-Context Learnercited by 39
  5. 2024AutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Modelscited by 34
  6. 2026Beyond IID: How General Are Tabular Foundation Models, Really?cited by 30
  7. 2018Similarity encoding for learning with dirty categorical variablescited by 29
  8. 2024Lost in Space: Probing Fine-grained Spatial Understanding in Vision and Language Resamplerscited by 13
  9. 2025TIME: TabPFN-Integrated Multimodal Engine for Robust Tabular-Image Learningcited by 13
  10. 2023Vectorizing string entries for data processing on tables: when are larger language models better?cited by 12
  11. 2026MultiModalPFN: Extending Prior-Data Fitted Networks for Multimodal Tabular Learningcited by 11
  12. 2025TabGemma: Text-Based Tabular ICL via LLM using Continued Pretraining and Retrievalcited by 7
  13. 2026STRABLE: Benchmarking Tabular Machine Learning with Stringscited by 6
  14. 2026RamanBench: A Large-Scale Benchmark for Machine Learning on Raman Spectroscopycited by 5
  15. 2026MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Imagecited by 4
  16. 2020Supervised dimensionality reduction by a Linear Discriminant Analysis on pre-trained CNN featurescited by 3
  17. 2025Factor Augmented Supervised Learning with Text Embeddingscited by 2
  18. 2025Linear Dimensionality Reduction for Word Embeddings in Tabular Data Classificationcited by 2
  19. 2026Enhancing Tabular Learners with Context-Aware Semantic Embeddingscited by 2
  20. 2026Supervised Dimensionality Reduction Revisited: Why LDA on Frozen CNN Features Deserves a Second Lookcited by 1
  21. 2026Modular Multimodal Classification Without Fine-Tuning: A Simple Compositional Approachcited by 1
  22. 2026TILE: Bridging LLMs and Table Foundation Models for Tabular Learning with Stringscited by 1

Built on by

Not cited yet.