# Effective Distillation of Table-based Reasoning Ability from LLMs

arXiv: https://arxiv.org/abs/2309.13182 · HTML: https://mlsanity.com/paper/2309.13182/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 50 of its 58 references that are in the map.
- First posted on arXiv 22 Sep 2023.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)
- Knowledge distillation / [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/)

## Builds on (most cited first, 30 of 50)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [Teaching Small Language Models to Reason](https://mlsanity.com/paper/2212.08410/) (2022)
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://mlsanity.com/paper/2004.14373/) (2020)
- [Specializing Smaller Language Models towards Multi-Step Reasoning](https://mlsanity.com/paper/2301.12726/) (2023)
- [TAPEX: Table Pre-training via Learning a Neural SQL Executor](https://mlsanity.com/paper/2107.07653/) (2021)
- [Binding Language Models in Symbolic Languages](https://mlsanity.com/paper/2210.02875/) (2022)
- [Challenges in Data-to-Document Generation](https://mlsanity.com/paper/1707.08052/) (2017)
- [Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning](https://mlsanity.com/paper/2301.13808/) (2023)

## Built on by (most cited first, 23 of 23)

- [Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future](https://mlsanity.com/paper/2309.15402/) (2023)
- [Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey](https://mlsanity.com/paper/2402.17944/) (2024)
- [A Survey of Table Reasoning with Large Language Models](https://mlsanity.com/paper/2402.08259/) (2024)
- [TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Data](https://mlsanity.com/paper/2401.13223/) (2024)
- [RealHiTBench: A Comprehensive Realistic Hierarchical Table Benchmark for Evaluating LLM-Based Table Analysis](https://mlsanity.com/paper/2506.13405/) (2025)
- [From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models](https://mlsanity.com/paper/2311.13063/) (2023)
- [SLIDE: A Framework Integrating Small and Large Language Models for Open-Domain Dialogues Evaluation](https://mlsanity.com/paper/2405.15924/) (2024)
- [Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework](https://mlsanity.com/paper/2406.17962/) (2024)
- [Improving Biomedical Abstractive Summarisation with Knowledge Aggregation from Citation Papers](https://mlsanity.com/paper/2310.15684/) (2023)
- [Uncovering Limitations of Large Language Models in Information Seeking from Tables](https://mlsanity.com/paper/2406.04113/) (2024)
- [Mitigating Spurious Correlations Between Question and Answer via Chain-of-Thought Correctness Perception Distillation](https://mlsanity.com/paper/2509.05602/) (2025)
- [A Cross-Attention Augmented Model for Event-Triggered Context-Aware Story Generation](https://mlsanity.com/paper/2311.11271/) (2023)
- [Train & Constrain: Phonologically Informed Tongue-Twister Generation from Topics and Paraphrases](https://mlsanity.com/paper/2403.13901/) (2024)
- [Improving Table Understanding with LLMs and Entity-Oriented Search](https://mlsanity.com/paper/2508.17028/) (2025)
- [Emphasising Structured Information: Integrating Abstract Meaning Representation into LLMs for Enhanced Open-Domain Dialogue Evaluation](https://mlsanity.com/paper/2404.01129/) (2024)
- [Planning for Success: Exploring LLM Long-term Planning Capabilities in Table Understanding](https://mlsanity.com/paper/2508.17005/) (2025)
- [Self-Correction Distillation for Structured Data Question Answering](https://mlsanity.com/paper/2511.07998/) (2025)
- [TREB: a BERT attempt for imputing tabular data imputation](https://mlsanity.com/paper/2410.00022/) (2024)
- [ITUNLP at SemEval-2025 Task 8: Question-Answering over Tabular Data: A Zero-Shot Approach using LLM-Driven Code Generation](https://mlsanity.com/paper/2508.00762/) (2025)
- [ACL Anthology Helper: A Tool to Retrieve and Manage Literature from ACL Anthology](https://mlsanity.com/paper/2310.20467/) (2023)
- [BioMNER: A Dataset for Biomedical Method Entity Recognition](https://mlsanity.com/paper/2406.20038/) (2024)
- [LLM-Enabled Multi-Agent Systems: Empirical Evaluation and Insights into Emerging Design Patterns & Paradigms](https://mlsanity.com/paper/2601.03328/) (2026)
- [Cross-Domain, Multi-Task Data-to-Text Generation without In-Domain Training Data](https://mlsanity.com/paper/2608.23391/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing tasks. However, their enormous parameter size and extremely high requirements for compute power pose challenges for their practical deployment. Recent research has revealed that specific capabilities of LLMs, such as numerical reasoning, can be transferred to smaller models through distillation. Some studies explore the potential of leveraging LLMs to perform table-based reasoning. However, there has been no prior work focusing on table reasoning skills in smaller models specifically tailored for scientific table-to-text generation tasks. In this paper, we propose a novel table-based reasoning distillation approach, with the aim of distilling LLMs into tailored smaller models. Our experimental results have shown that a 220 million parameter model (Flan-T5-base) fine-tuned using distilled data, not only achieves a significant improvement compared to traditionally fine-tuned baselines, but also surpasses specific LLMs on a scientific table-to-text generation dataset. Our code is available at https://github.com/Bernard-Yang/DistillTableCoT.