# Learning Adaptive Sampling and Reconstruction for Volume Visualization

arXiv: https://arxiv.org/abs/2007.10093 · HTML: https://mlsanity.com/paper/2007.10093/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2020.

## Built on by (most cited first, 14 of 14)

- [Fast Neural Representations for Direct Volume Rendering](https://mlsanity.com/paper/2112.01579/) (2021)
- [DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization](https://mlsanity.com/paper/2204.06504/) (2022)
- [Deep Hierarchical Super Resolution for Scientific Data](https://mlsanity.com/paper/2107.00462/) (2021)
- [FoVolNet: Fast Volume Rendering using Foveated Deep Neural Networks](https://mlsanity.com/paper/2209.09965/) (2022)
- [ParamsDrag: Interactive Parameter Space Exploration via Image-Space Dragging](https://mlsanity.com/paper/2407.14100/) (2024)
- [GSCache: Real-Time Radiance Caching for Volume Path Tracing using 3D Gaussian Splatting](https://mlsanity.com/paper/2507.19718/) (2025)
- [Uncertainty-Aware Deep Neural Representations for Visual Analysis of Vector Field Data](https://mlsanity.com/paper/2407.16119/) (2024)
- [Uncertainty-Informed Volume Visualization using Implicit Neural Representation](https://mlsanity.com/paper/2408.06018/) (2024)
- [Photon Field Networks for Dynamic Real-Time Volumetric Global Illumination](https://mlsanity.com/paper/2304.07338/) (2023)
- [Visual Analysis of Prediction Uncertainty in Neural Networks for Deep Image Synthesis](https://mlsanity.com/paper/2406.18545/) (2024)
- [Monocular Depth Decomposition of Semi-Transparent Volume Renderings](https://mlsanity.com/paper/2206.13282/) (2022)
- [A Comparative Study of Neural Surface Reconstruction for Scientific Visualization](https://mlsanity.com/paper/2407.20868/) (2024)
- [Compressive Modeling and Visualization of Multivariate Scientific Data using Implicit Neural Representation](https://mlsanity.com/paper/2510.15535/) (2025)
- [From Scalars to Time Series: Rethinking Implicit Neural Representations for Time-Varying Volumetric Data](https://mlsanity.com/paper/2607.20970/) (2026)

## Abstract (arXiv)

A central challenge in data visualization is to understand which data samples are required to generate an image of a data set in which the relevant information is encoded. In this work, we make a first step towards answering the question of whether an artificial neural network can predict where to sample the data with higher or lower density, by learning of correspondences between the data, the sampling patterns and the generated images. We introduce a novel neural rendering pipeline, which is trained end-to-end to generate a sparse adaptive sampling structure from a given low-resolution input image, and reconstructs a high-resolution image from the sparse set of samples. For the first time, to the best of our knowledge, we demonstrate that the selection of structures that are relevant for the final visual representation can be jointly learned together with the reconstruction of this representation from these structures. Therefore, we introduce differentiable sampling and reconstruction stages, which can leverage back-propagation based on supervised losses solely on the final image. We shed light on the adaptive sampling patterns generated by the network pipeline and analyze its use for volume visualization including isosurface and direct volume rendering.