# A Survey on LLM Inference-Time Self-Improvement

arXiv: https://arxiv.org/abs/2412.14352 · HTML: https://mlsanity.com/paper/2412.14352/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 13 of its 124 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Self-rewarding alignment and self-training](https://mlsanity.com/synth/self-rewarding-alignment-and-self-training/)

## Builds on (most cited first, 13 of 13)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Breaking the Softmax Bottleneck: A High-Rank RNN Language Model](https://mlsanity.com/paper/1711.03953/) (2017)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [Disclosure and Mitigation of Gender Bias in LLMs](https://mlsanity.com/paper/2402.11190/) (2024)
- [Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models](https://mlsanity.com/paper/2412.02674/) (2024)
- [ReasoningRec: Bridging Personalized Recommendations and Human-Interpretable Explanations through LLM Reasoning](https://mlsanity.com/paper/2410.23180/) (2024)
- [The Stable Entropy Hypothesis and Entropy-Aware Decoding: An Analysis and Algorithm for Robust Natural Language Generation](https://mlsanity.com/paper/2302.06784/) (2023)

## Built on by (most cited first, 17 of 17)

- [LLM Post-Training: A Deep Dive into Reasoning Large Language Models](https://mlsanity.com/paper/2502.21321/) (2025)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [Reasoning Language Models: A Blueprint](https://mlsanity.com/paper/2501.11223/) (2025)
- [ConfTuner: Training Large Language Models to Express Their Confidence Verbally](https://mlsanity.com/paper/2508.18847/) (2025)
- [Agents of Change: Self-Evolving LLM Agents for Strategic Planning](https://mlsanity.com/paper/2506.04651/) (2025)
- [Will Pre-Training Ever End? A First Step Toward Next-Generation Foundation MLLMs via Self-Improving Systematic Cognition](https://mlsanity.com/paper/2503.12303/) (2025)
- [Self-Improvement in Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2510.02665/) (2025)
- [Neural Genetic Search in Discrete Spaces](https://mlsanity.com/paper/2502.10433/) (2025)
- [Self-Improvement of Large Language Models: A Technical Overview and Future Outlook](https://mlsanity.com/paper/2603.25681/) (2026)
- [TestNUC: Enhancing Test-Time Computing Approaches and Scaling through Neighboring Unlabeled Data Consistency](https://mlsanity.com/paper/2502.19163/) (2025)
- [ProtTeX: Structure-In-Context Reasoning and Editing of Proteins with Large Language Models](https://mlsanity.com/paper/2503.08179/) (2025)
- [A Solver-in-the-Loop Framework for Improving LLMs on Answer Set Programming for Logic Puzzle Solving](https://mlsanity.com/paper/2512.17093/) (2025)
- [SDA: Steering-Driven Distribution Alignment for Open LLMs without Fine-Tuning](https://mlsanity.com/paper/2511.16324/) (2025)
- [When Languages Disagree: Self-Evolving Multilingual LLM Judges](https://mlsanity.com/paper/2606.08092/) (2026)
- [Towards a Small Language Model Lifecycle Framework](https://mlsanity.com/paper/2506.07695/) (2025)
- [Textual Self-attention Network: Test-Time Preference Optimization through Textual Gradient-based Attention](https://mlsanity.com/paper/2511.06682/) (2025)
- [The Role of Disfluencies in Speech Translation](https://mlsanity.com/paper/2608.02138/) (2026)

## Abstract (arXiv)

Techniques that enhance inference through increased computation at test-time have recently gained attention. In this survey, we investigate the current state of LLM Inference-Time Self-Improvement from three different perspectives: Independent Self-improvement, focusing on enhancements via decoding or sampling methods; Context-Aware Self-Improvement, leveraging additional context or datastore; and Model-Aided Self-Improvement, achieving improvement through model collaboration. We provide a comprehensive review of recent relevant studies, contribute an in-depth taxonomy, and discuss challenges and limitations, offering insights for future research.