# Instance-Wise Adaptive Sampling for Dataset Construction in Approximating Inverse Problem Solutions

arXiv: https://arxiv.org/abs/2509.04583 · HTML: https://mlsanity.com/paper/2509.04583/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 47 references that are in the map.
- First posted on arXiv 4 Sep 2025.

## Builds on (most cited first, 23 of 23)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Scaling Data-Constrained Language Models](https://mlsanity.com/paper/2305.16264/) (2023)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [Generating Sequences by Learning to Self-Correct](https://mlsanity.com/paper/2211.00053/) (2022)
- [A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks](https://mlsanity.com/paper/2207.10289/) (2022)
- [Recursive Introspection: Teaching Language Model Agents How to Self-Improve](https://mlsanity.com/paper/2407.18219/) (2024)
- [Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling](https://mlsanity.com/paper/2502.06703/) (2025)
- [InversionNet: A Real-Time and Accurate Full Waveform Inversion with CNNs and continuous CRFs](https://mlsanity.com/paper/1811.07875/) (2018)
- [The Limits of Inference Scaling Through Resampling](https://mlsanity.com/paper/2411.17501/) (2024)
- [Scaling Laws For Deep Learning Based Image Reconstruction](https://mlsanity.com/paper/2209.13435/) (2022)
- [A Neural Network Warm-Start Approach for the Inverse Acoustic Obstacle Scattering Problem](https://mlsanity.com/paper/2212.08736/) (2022)
- [Bifidelity Parameter Estimation Using Conditional Diffusion Models](https://mlsanity.com/paper/2504.01894/) (2025)
- [Reinforced Inverse Scattering](https://mlsanity.com/paper/2206.04186/) (2022)
- [Large deviation theory-based adaptive importance sampling for rare events in high dimensions](https://mlsanity.com/paper/2209.06278/) (2022)
- [A Direct Sampling-Based Deep Learning Approach for Inverse Medium Scattering Problems](https://mlsanity.com/paper/2305.00250/) (2023)
- [Back-Projection Diffusion: Solving the Wideband Inverse Scattering Problem with Diffusion Models](https://mlsanity.com/paper/2408.02866/) (2024)

## Abstract (arXiv)

We propose an instance-wise adaptive sampling framework for constructing compact and informative training datasets for supervised learning of inverse problem solutions. Typical learning-based approaches aim to learn a general-purpose inverse map from datasets drawn from a prior distribution, with the training process independent of the specific test instance. When the prior has a high intrinsic dimension or when high accuracy of the learned solution is required, a large number of training samples may be needed, resulting in substantial data collection costs. In contrast, our method dynamically allocates sampling effort based on the specific test instance, enabling significant gains in sample efficiency. By iteratively refining the training dataset conditioned on the latest prediction, the proposed strategy tailors the dataset to the geometry of the inverse map around each test instance. We demonstrate the effectiveness of our approach in the inverse scattering problem under two types of structured priors. Our results show that the advantage of the adaptive method becomes more pronounced in settings with more complex priors or higher accuracy requirements. While our experiments focus on a particular inverse problem, the adaptive sampling strategy is broadly applicable and readily extends to other inverse problems, offering a scalable and practical alternative to conventional fixed-dataset training regimes.