# Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models

arXiv: https://arxiv.org/abs/2308.00304 · HTML: https://mlsanity.com/paper/2308.00304/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 92 references that are in the map.
- First posted on arXiv 1 Aug 2023.

## Builds on (most cited first, 30 of 41)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Solving Quantitative Reasoning Problems with Language Models](https://mlsanity.com/paper/2206.14858/) (2022)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://mlsanity.com/paper/2201.11990/) (2022)
- [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022)
- [Language Models are Multilingual Chain-of-Thought Reasoners](https://mlsanity.com/paper/2210.03057/) (2022)
- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [Complexity-Based Prompting for Multi-Step Reasoning](https://mlsanity.com/paper/2210.00720/) (2022)
- [Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought](https://mlsanity.com/paper/2210.01240/) (2022)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [Measuring Compositional Generalization: A Comprehensive Method on Realistic Data](https://mlsanity.com/paper/1912.09713/) (2019)
- [Teaching Small Language Models to Reason](https://mlsanity.com/paper/2212.08410/) (2022)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [Good-Enough Compositional Data Augmentation](https://mlsanity.com/paper/1904.09545/) (2019)

## Built on by (most cited first, 30 of 33)

- [Self-Discover: Large Language Models Self-Compose Reasoning Structures](https://mlsanity.com/paper/2402.03620/) (2024)
- [DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks](https://mlsanity.com/paper/2309.17167/) (2023)
- [LLM With Tools: A Survey](https://mlsanity.com/paper/2409.18807/) (2024)
- [Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning](https://mlsanity.com/paper/2403.02333/) (2024)
- [Determinants of LLM-assisted Decision-Making](https://mlsanity.com/paper/2402.17385/) (2024)
- [Graph-enhanced Large Language Models in Asynchronous Plan Reasoning](https://mlsanity.com/paper/2402.02805/) (2024)
- [MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoning](https://mlsanity.com/paper/2402.17231/) (2024)
- [DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph](https://mlsanity.com/paper/2406.17271/) (2024)
- [Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web](https://mlsanity.com/paper/2311.18751/) (2023)
- [Unleashing LLM Reasoning Capability via Scalable Question Synthesis from Scratch](https://mlsanity.com/paper/2410.18693/) (2024)
- [When does compositional structure yield compositional generalization? A kernel theory](https://mlsanity.com/paper/2405.16391/) (2024)
- [VipAct: Visual-Perception Enhancement via Specialized VLM Agent Collaboration and Tool-use](https://mlsanity.com/paper/2410.16400/) (2024)
- [Can Models Learn Skill Composition from Examples?](https://mlsanity.com/paper/2409.19808/) (2024)
- [StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving](https://mlsanity.com/paper/2311.08803/) (2023)
- [Limits of Transformer Language Models on Learning to Compose Algorithms](https://mlsanity.com/paper/2402.05785/) (2024)
- [Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs](https://mlsanity.com/paper/2405.15485/) (2024)
- [LaRS: Latent Reasoning Skills for Chain-of-Thought Reasoning](https://mlsanity.com/paper/2312.04684/) (2023)
- [When Reasoning Meets Information Aggregation: A Case Study with Sports Narratives](https://mlsanity.com/paper/2406.12084/) (2024)
- [SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Models](https://mlsanity.com/paper/2408.15565/) (2024)
- [Contextual Agentic Memory is a Memo, Not True Memory](https://mlsanity.com/paper/2604.27707/) (2026)
- [Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learning](https://mlsanity.com/paper/2506.10378/) (2025)
- [Skill Neologisms: Towards Skill-based Continual Learning](https://mlsanity.com/paper/2605.04970/) (2026)
- [Evaluating In-Context Learning of Libraries for Code Generation](https://mlsanity.com/paper/2311.09635/) (2023)
- [Modeling Complex Mathematical Reasoning via Large Language Model based MathAgent](https://mlsanity.com/paper/2312.08926/) (2023)
- [When Persona Attributes Improve Population Alignment in Large Language Models](https://mlsanity.com/paper/2609.02526/) (2026)
- [From Words to Worlds: Compositionality for Cognitive Architectures](https://mlsanity.com/paper/2407.13419/) (2024)
- [Real-time Adapting Routing (RAR): Improving Efficiency Through Continuous Learning in Software Powered by Layered Foundation Models](https://mlsanity.com/paper/2411.09837/) (2024)
- [Dynamic Skill Adaptation for Large Language Models](https://mlsanity.com/paper/2412.19361/) (2024)
- [FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in Large Language Models](https://mlsanity.com/paper/2407.01046/) (2024)
- [IEA-Plugin: An AI Agent Reasoner for Test Data Analytics](https://mlsanity.com/paper/2504.11496/) (2025)

## Abstract (arXiv)

We investigate how to elicit compositional generalization capabilities in large language models (LLMs). Compositional generalization empowers LLMs to solve complex problems by combining foundational skills, a critical reasoning ability akin to human intelligence. However, even the most advanced LLMs currently struggle with this form of reasoning. We examine this problem within the framework of in-context learning and find that demonstrating both foundational skills and compositional examples grounded in these skills within the same prompt context is crucial. We refer to this prompt structure as skills-in-context (SKiC). With as few as two exemplars, this in-context learning structure enables LLMs to tackle more challenging problems requiring innovative skill combinations, achieving near-perfect systematic generalization across a broad range of tasks. Intriguingly, SKiC also unlocks the latent potential of LLMs, allowing them to more actively utilize pre-existing internal skills acquired during earlier pretraining stages to solve complex reasoning problems. The SKiC structure is robust across different skill constructions and exemplar choices and demonstrates strong transferability to new tasks. Finally, inspired by our in-context learning study, we show that fine-tuning LLMs with SKiC-style data can elicit zero-shot weak-to-strong generalization, enabling the models to solve much harder problems directly with standard prompting.