# FollowTable: A Benchmark for Instruction-Following Table Retrieval

arXiv: https://arxiv.org/abs/2605.00400 · HTML: https://mlsanity.com/paper/2605.00400/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 38 of its 60 references that are in the map.
- First posted on arXiv 1 May 2026.

## Builds on (most cited first, 30 of 38)

- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://mlsanity.com/paper/2212.09741/) (2022)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [MinerU: An Open-Source Solution for Precise Document Content Extraction](https://mlsanity.com/paper/2409.18839/) (2024)
- [Open Question Answering over Tables and Text](https://mlsanity.com/paper/2010.10439/) (2020)
- [RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!](https://mlsanity.com/paper/2312.02724/) (2023)
- [ReasonIR: Training Retrievers for Reasoning Tasks](https://mlsanity.com/paper/2504.20595/) (2025)
- [Open Domain Question Answering over Tables via Dense Retrieval](https://mlsanity.com/paper/2103.12011/) (2021)
- [Rank1: Test-Time Compute for Reranking in Information Retrieval](https://mlsanity.com/paper/2502.18418/) (2025)
- [FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions](https://mlsanity.com/paper/2403.15246/) (2024)
- [Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.08826/) (2025)
- [Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models](https://mlsanity.com/paper/2409.11136/) (2024)
- [Is Table Retrieval a Solved Problem? Exploring Join-Aware Multi-Table Retrieval](https://mlsanity.com/paper/2404.09889/) (2024)
- [Retrieving Complex Tables with Multi-Granular Graph Representation Learning](https://mlsanity.com/paper/2105.01736/) (2021)
- [StruBERT: Structure-aware BERT for Table Search and Matching](https://mlsanity.com/paper/2203.14278/) (2022)
- [Exploring the Impact of Table-to-Text Methods on Augmenting LLM-based Question Answering with Domain Hybrid Data](https://mlsanity.com/paper/2402.12869/) (2024)
- [MAIR: A Massive Benchmark for Evaluating Instructed Retrieval](https://mlsanity.com/paper/2410.10127/) (2024)
- [Table Search Using a Deep Contextualized Language Model](https://mlsanity.com/paper/2005.09207/) (2020)
- [Ad Hoc Table Retrieval using Semantic Similarity](https://mlsanity.com/paper/1802.06159/) (2018)
- [TARGET: Benchmarking Table Retrieval for Generative Tasks](https://mlsanity.com/paper/2505.11545/) (2025)
- [IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in Expert-Domain Information Retrieval](https://mlsanity.com/paper/2503.04644/) (2025)
- [Web Table Extraction, Retrieval and Augmentation: A Survey](https://mlsanity.com/paper/2002.00207/) (2020)
- [Content-Based Table Retrieval for Web Queries](https://mlsanity.com/paper/1706.02427/) (2017)
- [Template-Based Financial Report Generation in Agentic and Decomposed Information Retrieval](https://mlsanity.com/paper/2504.14233/) (2025)

## Abstract (arXiv)

Table Retrieval (TR) has traditionally been formulated as an ad-hoc retrieval problem, where relevance is primarily determined by topical semantic similarity. With the growing adoption of LLM-based agentic systems, access to structured data is increasingly instruction-driven, where relevance is conditional on explicit content and schema constraints rather than topical similarity alone. We therefore formalize Instruction-Following Table Retrieval (IFTR), a new task that requires models to jointly satisfy topical relevance and fine-grained instruction constraints. We identify two core challenges in IFTR: (i) sensitivity to content scope, such as inclusion and exclusion constraints, and (ii) awareness of schema-grounded requirements, including column semantics and representation granularity--capabilities largely absent in existing retrievers. To support systematic evaluation, we introduce FollowTable, the first large-scale benchmark for IFTR, constructed via a taxonomy-driven annotation pipeline. We further propose a new metric, termed the Instruction Responsiveness Score, to evaluate whether retrieval rankings consistently adapt to user instructions relative to a topic-only baseline. Our results indicate that existing retrieval models struggle to follow fine-grained instructions over tabular data. In particular, they exhibit systematic biases toward surface-level semantic cues and remain limited in handling schema-grounded constraints, highlighting substantial room for future improvements.