mlsanityData to 14 Oct 2026

Learning to Reduce: Towards Improving Performance of Large Language Models on Structured Data

arXiv 2407.02750 ↗First posted 3 Jul 2024cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have been achieving competent performance on a wide range of downstream tasks, yet existing work shows that inference on structured data is challenging for LLMs. This is because LLMs need to either understand long structured data or select the most relevant evidence before inference, and both approaches are not trivial. This paper proposes a framework, Learning to Reduce, that fine-tunes a language model with On-Policy Learning to generate a reduced version of an input structured data. When compared to state-of-the-art LLMs like GPT-4, Learning to Reduce not only achieves outstanding performance in reducing the input, but shows generalizability on different datasets. We further show that the model fine-tuned with our framework helps LLMs better perform on table QA tasks especially when the context is longer.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 24 of its 27 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

24 of 27 references.

References · 24

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  11. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  12. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 515
12 more
  1. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 428
  2. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  3. 2023StructGPT: A General Framework for Large Language Model to Reason over Structured Datacited by 343
  4. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
  5. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 277
  6. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 264
  7. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  8. 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 178
  9. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  10. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 95
  11. 2023Can Large Language Models Understand Real-World Complex Instructions?cited by 69
  12. 2020On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queriescited by 60

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  2. 2025Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluationcited by 5
  3. 2025Towards Question Answering over Large Semi-structured Tablescited by 2