mlsanityData to 7 Oct 2026

Auto-ICL: In-Context Learning without Human Supervision

arXiv 2311.09263 ↗First posted 15 Nov 2023cs.LG cs.AI cs.CLcited by 18

Not on the map. Why?

Abstract (arXiv)

With in-context learning ability, the performance of large language models can be significantly boosted when provided with appropriate context. However, existing in-context learning methods mainly rely on human-provided contexts, such as labeled examples and explicit instructions. Writing context by humans is labor-intensive on various tasks and limits the model to tasks manageable by humans. To overcome these limitations, we propose Automatic In-Context Learning framework that enables the model to autonomously generate examples and instructions for problem-solving. With experiments across various models and datasets, results show that model-generated contexts outperform human-annotated contexts, including Few-Shot and Few-Shot-CoT methods, and surpass existing self-generated context methods like Zero-CoT and Auto-CoT.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 22 of its 43 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

22 of 43 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  7. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  8. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  11. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  12. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
10 more
  1. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  2. 2016Solving General Arithmetic Word Problemscited by 353
  3. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  4. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  5. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  6. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  7. 2023Large Language Models as Analogical Reasonerscited by 107
  8. 2022Self-Generated In-Context Learning: Leveraging Auto-regressive Language Models as a Demonstration Generatorcited by 68
  9. 2023PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documentscited by 61
  10. 2022Self-Prompting Large Language Models for Zero-Shot Open-Domain QAcited by 39

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 7; 2026: 3.

18 papers.

Sort:

By area:

  1. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  2. 2024Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmarkcited by 62
  3. 2025Emergent Abilities in Large Language Models: A Surveycited by 34
  4. 2025Where to show Demos in Your Prompt: A Positional Bias of In-Context Learningcited by 13
  5. 2025ContextNav: Towards Agentic Multimodal In-Context Learningcited by 9
  6. 2024Relevant or Random: Can LLMs Truly Perform Analogical Reasoning?cited by 8
  7. 2024Vocabulary-Defined Semantics: Latent Space Clustering for Improving In-Context Learningcited by 7
  8. 2024Revisiting In-Context Learning with Long Context Language Modelscited by 7
  9. 2025What Makes a Good Natural Language Prompt?cited by 6
  10. 2024Let's Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Modelscited by 4
  11. 2025Unlocking In-Context Learning for Natural Datasets Beyond Language Modellingcited by 4
  12. 2024In-Context Transfer Learning: Demonstration Synthesis by Transferring Similar Taskscited by 3
6 more
  1. 2024Document-Level Event Extraction with Definition-Driven ICLcited by 2
  2. 2024Inductive Linguistic Reasoning with Large Language Modelscited by 2
  3. 2026Unlabeled Data Can Provably Enhance In-Context Learning of Transformerscited by 2
  4. 2026AEMA: Verifiable Evaluation Framework for Trustworthy and Controlled Agentic LLM Systemscited by 1
  5. 2026Learning to Select Visual In-Context Demonstrationscited by 1
  6. 2025Large Language Models Can Perform Automatic Modulation Classification via Discretized Self-supervised Candidate Retrievalcited by 0