# Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs

arXiv: https://arxiv.org/abs/2410.08020 · HTML: https://mlsanity.com/paper/2410.08020/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2024.

## Built on by (most cited first, 30 of 42)

- [Reinforcement Learning via Self-Distillation](https://mlsanity.com/paper/2601.20802/) (2026)
- [The Surprising Effectiveness of Test-Time Training for Few-Shot Learning](https://mlsanity.com/paper/2411.07279/) (2024)
- [Self-Improving LLM Agents at Test-Time](https://mlsanity.com/paper/2510.07841/) (2025)
- [FineMedLM-o1: Enhancing Medical Knowledge Reasoning Ability of LLM from Supervised Fine-Tuning to Test-Time Training](https://mlsanity.com/paper/2501.09213/) (2025)
- [Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiences](https://mlsanity.com/paper/2509.16189/) (2025)
- [Post-hoc Probabilistic Vision-Language Models](https://mlsanity.com/paper/2412.06014/) (2024)
- [Continuous Self-Improvement of Large Language Models by Test-time Training with Verifier-Driven Sample Selection](https://mlsanity.com/paper/2505.19475/) (2025)
- [DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning](https://mlsanity.com/paper/2505.19850/) (2025)
- [Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler](https://mlsanity.com/paper/2510.27172/) (2025)
- [Local Mixtures of Experts: Essentially Free Test-Time Training via Model Merging](https://mlsanity.com/paper/2505.14136/) (2025)
- [Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning](https://mlsanity.com/paper/2510.04786/) (2025)
- [Test-time Offline Reinforcement Learning on Goal-related Experience](https://mlsanity.com/paper/2507.18809/) (2025)
- [The Signal is in the Steps: Local Scoring for Reasoning Data Selection](https://mlsanity.com/paper/2510.03988/) (2025)
- [Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Models](https://mlsanity.com/paper/2510.07632/) (2025)
- [Gated Differentiable Working Memory for Long-Context Language Modeling](https://mlsanity.com/paper/2601.12906/) (2026)
- [Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Models](https://mlsanity.com/paper/2509.24510/) (2025)
- [Reasoning-Driven Synthetic Data Generation and Evaluation](https://mlsanity.com/paper/2603.29791/) (2026)
- [RTTC: Reward-Guided Collaborative Test-Time Compute](https://mlsanity.com/paper/2508.10024/) (2025)
- [Test-Time Tuned Language Models Enable End-to-end De Novo Molecular Structure Generation from MS/MS Spectra](https://mlsanity.com/paper/2510.23746/) (2025)
- [AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?](https://mlsanity.com/paper/2608.00155/) (2026)
- [Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models](https://mlsanity.com/paper/2510.14853/) (2025)
- [Scaling Natural-Language Graph-Based Test Time Compute for Automated Theorem Proving](https://mlsanity.com/paper/2503.11657/) (2025)
- [Prompts to Proxies: Emulating Human Preferences via a Compact LLM Ensemble](https://mlsanity.com/paper/2509.11311/) (2025)
- [Llama-based source code vulnerability detection: Prompt engineering vs Fine tuning](https://mlsanity.com/paper/2512.09006/) (2025)
- [EDCO: Dynamic Curriculum Orchestration for Domain-specific Large Language Model Fine-tuning](https://mlsanity.com/paper/2601.03725/) (2026)
- [Efficient Test-Time Finetuning of LLMs via Convex Reconstruction and Gradient Caching](https://mlsanity.com/paper/2605.30337/) (2026)
- [HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualization](https://mlsanity.com/paper/2606.26614/) (2026)
- [Focus on Likely Classes for Test-Time Prediction](https://mlsanity.com/paper/2505.03819/) (2025)
- [QueueEDIT: Structural Self-Correction for Sequential Model Editing in LLMs](https://mlsanity.com/paper/2506.17864/) (2025)
- [ETT: Expanding the Long Context Understanding Capability of LLMs at Test-Time](https://mlsanity.com/paper/2507.06313/) (2025)

## Abstract (arXiv)

Recent efforts in fine-tuning language models often rely on automatic data selection, commonly using Nearest Neighbors retrieval from large datasets. However, we theoretically show that this approach tends to select redundant data, limiting its effectiveness or even hurting performance. To address this, we introduce SIFT, a data selection algorithm designed to reduce uncertainty about the model's response given a prompt, which unifies ideas from retrieval and active learning. Whereas Nearest Neighbor retrieval typically fails in the presence of information duplication, SIFT accounts for information duplication and optimizes the overall information gain of the selected examples. We focus our evaluations on fine-tuning at test-time for prompt-specific language modeling on the Pile dataset, and show that SIFT consistently outperforms Nearest Neighbor retrieval, with minimal computational overhead. Moreover, we show that our uncertainty estimates can predict the performance gain of test-time fine-tuning, and use this to develop an adaptive algorithm that invests test-time compute proportional to realized performance gains. We provide the textttactiveft (Active Fine-Tuning) library which can be used as a drop-in replacement for Nearest Neighbor retrieval.