# Tabular Embedding Model (TEM): Finetuning Embedding Models For Tabular RAG Applications

arXiv: https://arxiv.org/abs/2405.01585 · HTML: https://mlsanity.com/paper/2405.01585/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 17 references that are in the map.
- First posted on arXiv 28 Apr 2024.

## Builds on (most cited first, 8 of 8)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [RAFT: Adapting Language Model to Domain Specific RAG](https://mlsanity.com/paper/2403.10131/) (2024)
- [Open Question Answering over Tables and Text](https://mlsanity.com/paper/2010.10439/) (2020)
- [Testing LLMs on Code Generation with Varying Levels of Prompt Specificity](https://mlsanity.com/paper/2311.07599/) (2023)
- [Unlock the Potential of Large Language Models for Predictive Tabular Tasks in Data Science with Table-Specific Pretraining](https://mlsanity.com/paper/2403.20208/) (2024)

## Built on by (most cited first, 8 of 8)

- [LLM Embeddings for Deep Learning on Tabular Data](https://mlsanity.com/paper/2502.11596/) (2025)
- [QAEncoder: Towards Aligned Representation Learning in Question Answering Systems](https://mlsanity.com/paper/2409.20434/) (2024)
- [A Knowledge Graph and a Tripartite Evaluation Framework Make Retrieval-Augmented Generation Scalable and Transparent](https://mlsanity.com/paper/2509.19209/) (2025)
- [Statistical Embeddings for Similarity, Retrieval, and Interpretable Alignment of Numeric Tabular Datasets](https://mlsanity.com/paper/2605.30289/) (2026)
- [Innovative tokenisation of structured data for LLM training](https://mlsanity.com/paper/2508.01685/) (2025)
- [Retrieval-Augmented Generation to Support Railways Engineering Tasks: A Case Study](https://mlsanity.com/paper/2607.01244/) (2026)
- [Semantically Similar, Logically Distinct: Diagnosing the Semantic-Answerability Gap in Table RAG](https://mlsanity.com/paper/2607.17742/) (2026)
- [TableSeek: Structure-Preserving Agentic Evidence Seeking over Heterogeneous Table Corpora](https://mlsanity.com/paper/2609.34157/) (2026)

## Abstract (arXiv)

In recent times Large Language Models have exhibited tremendous capabilities, especially in the areas of mathematics, code generation and general-purpose reasoning. However for specialized domains especially in applications that require parsing and analyzing large chunks of numeric or tabular data even state-of-the-art (SOTA) models struggle. In this paper, we introduce a new approach to solving domain-specific tabular data analysis tasks by presenting a unique RAG workflow that mitigates the scalability issues of existing tabular LLM solutions. Specifically, we present Tabular Embedding Model (TEM), a novel approach to fine-tune embedding models for tabular Retrieval-Augmentation Generation (RAG) applications. Embedding models form a crucial component in the RAG workflow and even current SOTA embedding models struggle as they are predominantly trained on textual datasets and thus underperform in scenarios involving complex tabular data. The evaluation results showcase that our approach not only outperforms current SOTA embedding models in this domain but also does so with a notably smaller and more efficient model structure.