# VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning

arXiv: https://arxiv.org/abs/2605.08146 · HTML: https://mlsanity.com/paper/2605.08146/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 3 May 2026.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [StructGPT: A General Framework for Large Language Model to Reason over Structured Data](https://mlsanity.com/paper/2305.09645/) (2023)
- [The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink](https://mlsanity.com/paper/2204.05149/) (2022)
- [MultiModalQA: Complex Question Answering over Text, Tables and Images](https://mlsanity.com/paper/2104.06039/) (2021)
- [Pixtral 12B](https://mlsanity.com/paper/2410.07073/) (2024)
- [Thyme: Think Beyond Images](https://mlsanity.com/paper/2508.11630/) (2025)
- [Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data](https://mlsanity.com/paper/2303.14080/) (2023)
- [TableVQA-Bench: A Visual Question Answering Benchmark on Multiple Table Domains](https://mlsanity.com/paper/2404.19205/) (2024)
- [EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Images](https://mlsanity.com/paper/2310.18652/) (2023)
- [Multimodal Table Understanding](https://mlsanity.com/paper/2406.08100/) (2024)
- [Combining 3D Image and Tabular Data via the Dynamic Affine Feature Map Transform](https://mlsanity.com/paper/2107.05990/) (2021)
- [Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Review](https://mlsanity.com/paper/2203.15588/) (2022)
- [RadFusion: Benchmarking Performance and Fairness for Multimodal Pulmonary Embolism Detection from CT and EHR](https://mlsanity.com/paper/2111.11665/) (2021)
- [TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Data](https://mlsanity.com/paper/2407.07582/) (2024)
- [Learngene: Inheriting Condensed Knowledge from the Ancestry Model to Descendant Models](https://mlsanity.com/paper/2305.02279/) (2023)
- [DVM-CAR: A large-scale automotive dataset for visual marketing research and applications](https://mlsanity.com/paper/2109.00881/) (2021)
- [Multimodal Tabular Reasoning with Privileged Structured Information](https://mlsanity.com/paper/2506.04088/) (2025)
- [MMR-Bench: A Comprehensive Benchmark for Multimodal LLM Routing](https://mlsanity.com/paper/2601.17814/) (2026)
- [Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models](https://mlsanity.com/paper/2404.16897/) (2024)
- [Knowledge-Aware Reasoning over Multimodal Semi-structured Tables](https://mlsanity.com/paper/2408.13860/) (2024)
- [MMTABREAL: Real-World Benchmark for Multimodal Table Understanding](https://mlsanity.com/paper/2505.21771/) (2025)
- [MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space](https://mlsanity.com/paper/2506.11684/) (2025)
- [TopBench: A Benchmark for Implicit Predictive Reasoning in Tabular Question Answering](https://mlsanity.com/paper/2604.28076/) (2026)

## Built on by (most cited first, 3 of 3)

- [MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image](https://mlsanity.com/paper/2605.10616/) (2026)
- [On the Learnability of Test-Time Adaptation: A Recovery Complexity Perspective](https://mlsanity.com/paper/2605.28057/) (2026)
- [SymboLLM-FE: LLM-Accelerated Symbolic Regression for Automated Feature Engineering on Tabular Data](https://mlsanity.com/paper/2608.28408/) (2026)

## Abstract (arXiv)

Multi-model learning has attracted great attention in visual-text tasks. However, visual-tabular data, which plays a pivotal role in high-stakes domains like healthcare and industry, remains underexplored. In this paper, we introduce textitVT-Bench, the first unified benchmark for standardizing vision-tabular discriminative prediction and generative reasoning tasks. VT-Bench aggregates 14 datasets across 9 domains (medical-centric, while covering pets, media, and transportation) with over 756K samples. We evaluate 23 representative models, including unimodal experts, specialized visual-tabular models, general-purpose vision-language models (VLMs), and tool-augmented methods, highlighting substantial challenges of visual-tabular learning. We believe VT-Bench will stimulate the community to build more powerful multi-modal vision-tabular foundation models. Benchmark: https://github.com/Ziyi-Jia990/VT-Bench