# An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models

arXiv: https://arxiv.org/abs/2401.06692 · HTML: https://mlsanity.com/paper/2401.06692/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 59 references that are in the map.
- First posted on arXiv 12 Jan 2024.

## Builds on (most cited first, 30 of 36)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks](https://mlsanity.com/paper/2204.07705/) (2022)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources](https://mlsanity.com/paper/2306.04751/) (2023)
- [Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor](https://mlsanity.com/paper/2212.09689/) (2022)
- [Selection via Proxy: Efficient Data Selection for Deep Learning](https://mlsanity.com/paper/1906.11829/) (2019)
- [From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning](https://mlsanity.com/paper/2308.12032/) (2023)
- [Efficiently Identifying Task Groupings for Multi-Task Learning](https://mlsanity.com/paper/2109.04617/) (2021)
- [Adversarial Active Learning for Deep Networks: a Margin Based Approach](https://mlsanity.com/paper/1802.09841/) (2018)
- [Batch Active Learning at Scale](https://mlsanity.com/paper/2107.14263/) (2021)
- [MoDS: Model-oriented Data Selection for Instruction Tuning](https://mlsanity.com/paper/2311.15653/) (2023)
- [Lazier Than Lazy Greedy](https://mlsanity.com/paper/1409.7938/) (2014)
- [Gone Fishing: Neural Active Learning with Fisher Embeddings](https://mlsanity.com/paper/2106.09675/) (2021)
- [Deep Active Learning over the Long Tail](https://mlsanity.com/paper/1711.00941/) (2017)
- [Data Diversity Matters for Robust Instruction Tuning](https://mlsanity.com/paper/2311.14736/) (2023)
- [Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks](https://mlsanity.com/paper/2311.00288/) (2023)
- [ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization](https://mlsanity.com/paper/2201.06910/) (2022)
- [GALAXY: Graph-based Active Learning at the Extreme](https://mlsanity.com/paper/2202.01402/) (2022)
- [CiT: Curation in Training for Effective Vision-Language Data](https://mlsanity.com/paper/2301.02241/) (2023)
- [LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning](https://mlsanity.com/paper/2306.09910/) (2023)

## Built on by (most cited first, 23 of 23)

- [The Best Instruction-Tuning Data are Those That Fit](https://mlsanity.com/paper/2502.04194/) (2025)
- [SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models](https://mlsanity.com/paper/2403.07384/) (2024)
- [Diversity Measurement and Subset Selection for Instruction Tuning Datasets](https://mlsanity.com/paper/2402.02318/) (2024)
- [Efficient Data Selection at Scale via Influence Distillation](https://mlsanity.com/paper/2505.19051/) (2025)
- [Offline Multi-task Transfer RL with Representational Penalization](https://mlsanity.com/paper/2402.12570/) (2024)
- [DELIFT: Data Efficient Language model Instruction Fine Tuning](https://mlsanity.com/paper/2411.04425/) (2024)
- [Mini-batch Coresets for Memory-efficient Language Model Training on Data Mixtures](https://mlsanity.com/paper/2407.19580/) (2024)
- [Data Selection for Fine-tuning Vision Language Models via Cross Modal Alignment Trajectories](https://mlsanity.com/paper/2510.01454/) (2025)
- [Deep Active Learning in the Open World](https://mlsanity.com/paper/2411.06353/) (2024)
- [Active Testing of Large Language Models via Approximate Neyman Allocation](https://mlsanity.com/paper/2605.10075/) (2026)
- [ParetoHqD: Fast Offline Multiobjective Alignment of Large Language Models using Pareto High-quality Data](https://mlsanity.com/paper/2504.16628/) (2025)
- [Towards Multimodal Active Learning: Efficient Learning with Limited Paired Data](https://mlsanity.com/paper/2510.03247/) (2025)
- [Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models](https://mlsanity.com/paper/2408.15915/) (2024)
- [COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation](https://mlsanity.com/paper/2412.17684/) (2024)
- [Mixtraining: A Better Trade-Off Between Compute and Performance](https://mlsanity.com/paper/2502.19513/) (2025)
- [Deep Submodular Peripteral Networks](https://mlsanity.com/paper/2403.08199/) (2024)
- [Density-Aware Farthest Point Sampling](https://mlsanity.com/paper/2509.13213/) (2025)
- [Seeing is Believing (and Predicting): Context-Aware Multi-Human Behavior Prediction with Vision Language Models](https://mlsanity.com/paper/2512.15957/) (2025)
- [Online Finetuning Decision Transformers with Pure RL Gradients](https://mlsanity.com/paper/2601.00167/) (2026)
- [Experimental Design for Active Transductive Inference in Large Language Models](https://mlsanity.com/paper/2404.08846/) (2024)
- [Beyond Expected Information Gain: Stable Bayesian Optimal Experimental Design with Integral Probability Metrics and Plug-and-Play Extensions](https://mlsanity.com/paper/2604.21849/) (2026)
- [Active Learning with Foundation Model Priors: Efficient Learning under Class Imbalance](https://mlsanity.com/paper/2606.07630/) (2026)
- [LLM Evaluation is a Forecasting Problem](https://mlsanity.com/paper/2610.17018/) (2026)

## Abstract (arXiv)

Supervised finetuning (SFT) on instruction datasets has played a crucial role in achieving the remarkable zero-shot generalization capabilities observed in modern large language models (LLMs). However, the annotation efforts required to produce high quality responses for instructions are becoming prohibitively expensive, especially as the number of tasks spanned by instruction datasets continues to increase. Active learning is effective in identifying useful subsets of samples to annotate from an unlabeled pool, but its high computational cost remains a barrier to its widespread applicability in the context of LLMs. To mitigate the annotation cost of SFT and circumvent the computational bottlenecks of active learning, we propose using experimental design. Experimental design techniques select the most informative samples to label, and typically maximize some notion of uncertainty and/or diversity. In our work, we implement a framework that evaluates several existing and novel experimental design techniques and find that these methods consistently yield significant gains in label efficiency with little computational overhead. On generative tasks, our methods achieve the same generalization performance with only 50% of annotation cost required by random sampling.