mlsanityData to 7 Oct 2026

Augmenting In-Context-Learning in LLMs via Automatic Data Labeling and Refinement

arXiv 2410.10348 ↗First posted 14 Oct 2024cs.CLcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

It has been shown that Large Language Models' (LLMs) performance can be improved for many tasks using Chain of Thought (CoT) or In-Context Learning (ICL), which involve demonstrating the steps needed to solve a task using a few examples. However, while datasets with input-output pairs are relatively easy to produce, providing demonstrations which include intermediate steps requires cumbersome manual work. These steps may be executable programs, as in agentic flows, or step-by-step reasoning as in CoT. In this work, we propose Automatic Data Labeling and Refinement (ADLR), a method to automatically generate and filter demonstrations which include the above intermediate steps, starting from a small seed of manually crafted examples. We demonstrate the advantage of ADLR in code-based table QA and mathematical reasoning, achieving up to a 5.5% gain. The code implementing our method is provided in the Supplementary material and will be made available.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 25 of its 32 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

25 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Models6 Oct

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  7. 2016Matching Networks for One Shot Learningcited by 2,246
  8. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  9. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  10. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  11. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  12. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
12 more
  1. 2019TabFact: A Large-scale Dataset for Table-based Fact Verificationcited by 425
  2. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  3. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  4. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  5. 2022Binding Language Models in Symbolic Languagescited by 224
  6. 2022Active Example Selection for In-Context Learningcited by 195
  7. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 185
  8. 2023Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoningcited by 177
  9. 2022Data Curation Alone Can Stabilize In-context Learningcited by 58
  10. 2022ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examplescited by 29
  11. 2023RetICL: Sequential Retrieval of In-Context Examples with Reinforcement Learningcited by 24
  12. 2024CABINET: Content Relevance based Noise Reduction for Table Question Answeringcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026NTILC: Neural Tool Invocation via Learned Compressioncited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Process In-Context Learning: Enhancing Mathematical Reasoning via Dynamic Demonstration Insertion
  2. 2024What Makes In-context Learning Effective for Mathematical Reasoning: A Theoretical Analysis
  3. 2024Demonstration Notebook: Finding the Most Suited In-Context Learning Example from Interactions
  4. 2025Leveraging In-Context Learning for Language Model Agents
  5. 2026ReasonCACHE: Teaching LLMs To Reason Without Weight Updates
  6. 2023ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought
  7. 2024LLM With Tools: A Survey
  8. 2025Prompting Test-Time Scaling Is A Strong LLM Reasoning Data Augmentation