mlsanityData to 7 Oct 2026

Structured Prompting: Scaling In-Context Learning to 1,000 Examples

arXiv 2212.06713 ↗First posted 13 Dec 2022cs.CLcited by 81

Not on the map. Why?

Abstract (arXiv)

Large language models have exhibited intriguing in-context learning capability, achieving promising zero- and few-shot performance without updating the parameters. However, conventional in-context learning is usually restricted by length constraints, rendering it ineffective to absorb supervision from a large number of examples. In order to go beyond few shots, we introduce structured prompting that breaks the length limit and scales in-context learning to thousands of examples. Specifically, demonstration examples are separately encoded with well-designed position embeddings, and then they are jointly attended by the test example using a rescaled attention mechanism. So we can scale the number of exemplars with linear complexity instead of quadratic complexity with respect to length. Experimental results on a diverse set of tasks show that our approach improves end-task performance and reduces evaluation variance over conventional in-context learning as the number of demonstration examples increases. Code has been released at https://aka.ms/structured-prompting.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 13 Dec 2022.

Builds on

30 of 44 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  12. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
18 more
  1. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  2. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  3. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  4. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  5. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  6. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  7. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  8. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  9. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  10. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  11. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  12. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  13. 2021Efficient Large Scale Language Modeling with Mixtures of Expertscited by 175
  14. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  15. 2022Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrationscited by 116
  16. 2022Language Models are General-Purpose Interfacescited by 96
  17. 2022On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Modelcited by 78
  18. 2022Prototypical Calibration for Few-shot Learning of Language Modelscited by 58

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 18; 2024: 31; 2025: 24; 2026: 6.

81 papers.

Sort:

By area:

  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  2. 2022A Survey on In-context Learningcited by 717
  3. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
  4. 2024Long-context LLMs Struggle with Long In-context Learningcited by 285
  5. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  6. 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 179
  7. 2023Focused Transformer: Contrastive Training for Context Scalingcited by 133
  8. 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 126
  9. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
  10. 2024Long-Context Language Modeling with Parallel Context Encodingcited by 84
  11. 2022Parallel Context Windows for Large Language Modelscited by 81
  12. 2023Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Surveycited by 71
38 more
  1. 2024Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Modelscited by 71
  2. 2024Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrationcited by 70
  3. 2024Eliminating Position Bias of Language Models: A Mechanistic Approachcited by 61
  4. 2023Learning to Retrieve In-Context Examples for Large Language Modelscited by 60
  5. 2024SCBench: A KV Cache-Centric Analysis of Long-Context Methodscited by 59
  6. 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 56
  7. 2023Pre-Training to Learn in Contextcited by 41
  8. 2025Emergent Abilities in Large Language Models: A Surveycited by 34
  9. 2023In-Context Learning for Attention Scheme: from Single Softmax Regression to Multiple Softmax Regression via a Tensor Trickcited by 32
  10. 2024Mixture of In-Context Prompters for Tabular PFNscited by 32
  11. 2024PDE Generalization of In-Context Operator Networks: A Study on 1D Scalar Nonlinear Conservation Lawscited by 30
  12. 2023Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruningcited by 26
  13. 2024Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Modelscited by 26
  14. 2024CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memorycited by 24
  15. 2023Language models are weak learnerscited by 21
  16. 2025From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generationcited by 20
  17. 2023Efficient Prompting via Dynamic In-Context Learningcited by 18
  18. 2023Iterative Forward Tuning Boosts In-Context Learning in Language Modelscited by 15
  19. 2024MEND: Meta dEmonstratioN Distillation for Efficient and Effective In-Context Learningcited by 15
  20. 2025Where to show Demos in Your Prompt: A Positional Bias of In-Context Learningcited by 13
  21. 2023Design of Chain-of-Thought in Math Problem Solvingcited by 12
  22. 2025More is not always better? Enhancing Many-Shot In-Context Learning with Differentiated and Reweighting Objectivescited by 12
  23. 2024Implicit In-context Learningcited by 11
  24. 2024Demonstration Augmentation for Zero-shot In-context Learningcited by 11
  25. 2024Mitigating Copy Bias in In-Context Learning through Neuron Pruningcited by 11
  26. 2025A Survey on Transformer Context Extension: Approaches and Evaluationcited by 11
  27. 2025ZeroTuning: Unlocking the Initial Token's Power to Enhance Large Language Models Without Trainingcited by 11
  28. 2023Contextual Vision Transformers for Robust Representation Learningcited by 10
  29. 2024SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimizationcited by 10
  30. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  31. 2024Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learningcited by 9
  32. 2025ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attackscited by 9
  33. 2023Scaling In-Context Demonstrations with Structured Attentioncited by 8
  34. 2024Focused Large Language Models are Stable Many-Shot Learnerscited by 8
  35. 2024Enhancing In-Context Learning via Implicit Demonstration Augmentationcited by 6
  36. 2024SPICA: Retrieving Scenarios for Pluralistic In-Context Alignmentcited by 6
  37. 2024Naive Bayes-based Context Extension for Large Language Modelscited by 5
  38. 2024LIVE: Learnable In-Context Vector for Visual Question Answeringcited by 5

50 most cited of 81 All 81 (JSON)