# Evaluating LLMs on Entity Disambiguation in Tables

arXiv: https://arxiv.org/abs/2408.06423 · HTML: https://mlsanity.com/paper/2408.06423/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 77 references that are in the map.
- First posted on arXiv 12 Aug 2024.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Large Language Models Struggle to Learn Long-Tail Knowledge](https://mlsanity.com/paper/2211.08411/) (2022)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [Understanding the Effects of RLHF on LLM Generalisation and Diversity](https://mlsanity.com/paper/2310.06452/) (2023)
- [LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models](https://mlsanity.com/paper/2309.12307/) (2023)
- [Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding](https://mlsanity.com/paper/2401.04398/) (2024)
- [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024)
- [TableLlama: Towards Open Large Generalist Models for Tables](https://mlsanity.com/paper/2311.09206/) (2023)
- [Simple and Scalable Strategies to Continually Pre-train Large Language Models](https://mlsanity.com/paper/2403.08763/) (2024)
- [Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study](https://mlsanity.com/paper/2305.13062/) (2023)
- [Can ChatGPT replace StackOverflow? A Study on Robustness and Reliability of Large Language Model Code Generation](https://mlsanity.com/paper/2308.10335/) (2023)
- [Table-GPT: Table-tuned GPT for Diverse Table Tasks](https://mlsanity.com/paper/2310.09263/) (2023)
- [TabSQLify: Enhancing Reasoning Capabilities of LLMs Through Table Decomposition](https://mlsanity.com/paper/2404.10150/) (2024)
- [Annotating Columns with Pre-trained Language Models](https://mlsanity.com/paper/2104.01785/) (2021)
- [OpenTab: Advancing Large Language Models as Open-domain Table Reasoners](https://mlsanity.com/paper/2402.14361/) (2024)
- [LLMEval: A Preliminary Study on How to Evaluate Large Language Models](https://mlsanity.com/paper/2312.07398/) (2023)
- [Using ChatGPT for Entity Matching](https://mlsanity.com/paper/2305.03423/) (2023)
- [Column Type Annotation using ChatGPT](https://mlsanity.com/paper/2306.00745/) (2023)
- [Novel Entity Discovery from Web Tables](https://mlsanity.com/paper/2002.00206/) (2020)
- [MTab: Matching Tabular Data to Knowledge Graph using Probability Models](https://mlsanity.com/paper/1910.00246/) (2019)
- [Extracting Novel Facts from Tables for Knowledge Graph Completion (Extended version)](https://mlsanity.com/paper/1907.00083/) (2019)

## Built on by (most cited first, 2 of 2)

- [Survey on Semantic Interpretation of Tabular Data: Challenges and Directions](https://mlsanity.com/paper/2411.11891/) (2024)
- [Efficient Uncertainty Estimation for LLM-based Entity Linking in Tabular Data](https://mlsanity.com/paper/2510.01251/) (2025)

## Abstract (arXiv)

Tables are crucial containers of information, but understanding their meaning may be challenging. Over the years, there has been a surge in interest in data-driven approaches based on deep learning that have increasingly been combined with heuristic-based ones. In the last period, the advent of acfllms has led to a new category of approaches for table annotation. However, these approaches have not been consistently evaluated on a common ground, making evaluation and comparison difficult. This work proposes an extensive evaluation of four STI SOTA approaches: Alligator (formerly s-elbat), Dagobah, TURL, and TableLlama; the first two belong to the family of heuristic-based algorithms, while the others are respectively encoder-only and decoder-only Large Language Models (LLMs). We also include in the evaluation both GPT-4o and GPT-4o-mini, since they excel in various public benchmarks. The primary objective is to measure the ability of these approaches to solve the entity disambiguation task with respect to both the performance achieved on a common-ground evaluation setting and the computational and cost requirements involved, with the ultimate aim of charting new research paths in the field.