# DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization

arXiv: https://arxiv.org/abs/2204.06504 · HTML: https://mlsanity.com/paper/2204.06504/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 100 of its 189 references that are in the map.
- First posted on arXiv 13 Apr 2022.

## Builds on (most cited first, 30 of 100)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)

## Built on by (most cited first, 30 of 32)

- [ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules](https://mlsanity.com/paper/2304.02173/) (2023)
- [Generative AI for Visualization: State of the Art and Future Directions](https://mlsanity.com/paper/2404.18144/) (2024)
- [NLI4VolVis: Natural Language Interaction for Volume Visualization via LLM Multi-Agents and Editable 3D Gaussian Splatting](https://mlsanity.com/paper/2507.12621/) (2025)
- [HyperINR: A Fast and Predictive Hypernetwork for Implicit Neural Representations via Knowledge Distillation](https://mlsanity.com/paper/2304.04188/) (2023)
- [iVR-GS: Inverse Volume Rendering for Explorable Visualization via Editable 3D Gaussian Splatting](https://mlsanity.com/paper/2504.17954/) (2025)
- [ECNR: Efficient Compressive Neural Representation of Time-Varying Volumetric Datasets](https://mlsanity.com/paper/2311.12831/) (2023)
- [Adaptively Placed Multi-Grid Scene Representation Networks for Large-Scale Data Visualization](https://mlsanity.com/paper/2308.02494/) (2023)
- [Meta-INR: Efficient Encoding of Volumetric Data via Meta-Learning Implicit Neural Representation](https://mlsanity.com/paper/2502.09669/) (2025)
- [SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents](https://mlsanity.com/paper/2603.29139/) (2026)
- [StyleRF-VolVis: Style Transfer of Neural Radiance Fields for Expressive Volume Visualization](https://mlsanity.com/paper/2408.00150/) (2024)
- [TexGS-VolVis: Expressive Scene Editing for Volume Visualization via Textured Gaussian Splatting](https://mlsanity.com/paper/2507.13586/) (2025)
- [ParamsDrag: Interactive Parameter Space Exploration via Image-Space Dragging](https://mlsanity.com/paper/2407.14100/) (2024)
- [FCNR: Fast Compressive Neural Representation of Visualization Images](https://mlsanity.com/paper/2407.16369/) (2024)
- [FFEINR: Flow Feature-Enhanced Implicit Neural Representation for Spatio-temporal Super-Resolution](https://mlsanity.com/paper/2308.12508/) (2023)
- [Accelerated Probabilistic Marching Cubes by Deep Learning for Time-Varying Scalar Ensembles](https://mlsanity.com/paper/2207.07260/) (2022)
- [Uncertainty-Aware Deep Neural Representations for Visual Analysis of Vector Field Data](https://mlsanity.com/paper/2407.16119/) (2024)
- [Uncertainty-Informed Volume Visualization using Implicit Neural Representation](https://mlsanity.com/paper/2408.06018/) (2024)
- [CD-TVD: Contrastive Diffusion for 3D Super-Resolution with Scarce High-Resolution Time-Varying Data](https://mlsanity.com/paper/2508.08173/) (2025)
- [ECoNGS: Efficient Compressive Neural Gaussian Splats for Volume Visualization](https://mlsanity.com/paper/2607.18466/) (2026)
- [Photon Field Networks for Dynamic Real-Time Volumetric Global Illumination](https://mlsanity.com/paper/2304.07338/) (2023)
- [Visual Analysis of Prediction Uncertainty in Neural Networks for Deep Image Synthesis](https://mlsanity.com/paper/2406.18545/) (2024)
- [Adversarial Attacks on Machine Learning-Aided Visualizations](https://mlsanity.com/paper/2409.02485/) (2024)
- [HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualization](https://mlsanity.com/paper/2606.26614/) (2026)
- [Efficient Compression of Structured and Unstructured Volumes via Learned 3D Gaussian Representation](https://mlsanity.com/paper/2607.01164/) (2026)
- [Monocular Depth Decomposition of Semi-Transparent Volume Renderings](https://mlsanity.com/paper/2206.13282/) (2022)
- [Topology Guidance: Controlling the Outputs of Generative Models via Vector Field Topology](https://mlsanity.com/paper/2505.06804/) (2025)
- [Fixed-Budget Gaussian Volume Encoding with Structure-Aware Allocation](https://mlsanity.com/paper/2608.14112/) (2026)
- [VortexViz: Finding Vortex Boundaries by Learning from Particle Trajectories](https://mlsanity.com/paper/2404.01352/) (2024)
- [A Comparative Study of Neural Surface Reconstruction for Scientific Visualization](https://mlsanity.com/paper/2407.20868/) (2024)
- [Compressive Modeling and Visualization of Multivariate Scientific Data using Implicit Neural Representation](https://mlsanity.com/paper/2510.15535/) (2025)

## Abstract (arXiv)

Since 2016, we have witnessed the tremendous growth of artificial intelligence+visualization (AI+VIS) research. However, existing survey papers on AI+VIS focus on visual analytics and information visualization, not scientific visualization (SciVis). In this paper, we survey related deep learning (DL) works in SciVis, specifically in the direction of DL4SciVis: designing DL solutions for solving SciVis problems. To stay focused, we primarily consider works that handle scalar and vector field data but exclude mesh data. We classify and discuss these works along six dimensions: domain setting, research task, learning type, network architecture, loss function, and evaluation metric. The paper concludes with a discussion of the remaining gaps to fill along the discussed dimensions and the grand challenges we need to tackle as a community. This state-of-the-art survey guides SciVis researchers in gaining an overview of this emerging topic and points out future directions to grow this research.