# DeepOrganNet: On-the-Fly Reconstruction and Visualization of 3D / 4D Lung Models from Single-View Projections by Deep Deformation Network

arXiv: https://arxiv.org/abs/1907.09375 · HTML: https://mlsanity.com/paper/1907.09375/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2019.

## Built on by (most cited first, 15 of 15)

- [A Deep-Learning Approach For Direct Whole-Heart Mesh Reconstruction](https://mlsanity.com/paper/2102.07899/) (2021)
- [DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualization](https://mlsanity.com/paper/2204.06504/) (2022)
- [Generative AI for Visualization: State of the Art and Future Directions](https://mlsanity.com/paper/2404.18144/) (2024)
- [4D Myocardium Reconstruction with Decoupled Motion and Shape Model](https://mlsanity.com/paper/2308.14083/) (2023)
- [Image-to-Graph Convolutional Network for Deformable Shape Reconstruction from a Single Projection Image](https://mlsanity.com/paper/2108.12533/) (2021)
- [IGCN: Image-to-graph Convolutional Network for 2D/3D Deformable Registration](https://mlsanity.com/paper/2111.00484/) (2021)
- [3D Coronary Vessel Reconstruction from Bi-Plane Angiography using Graph Convolutional Networks](https://mlsanity.com/paper/2302.14795/) (2023)
- [From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction](https://mlsanity.com/paper/2505.03599/) (2025)
- [COVID-view: Diagnosis of COVID-19 using Chest CT](https://mlsanity.com/paper/2108.03799/) (2021)
- [Adversarial Attacks on Machine Learning-Aided Visualizations](https://mlsanity.com/paper/2409.02485/) (2024)
- [voxel2vec: A Natural Language Processing Approach to Learning Distributed Representations for Scientific Data](https://mlsanity.com/paper/2207.02565/) (2022)
- [Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Art](https://mlsanity.com/paper/2311.10129/) (2023)
- [Deformable Image Registration using Unsupervised Deep Learning for CBCT-guided Abdominal Radiotherapy](https://mlsanity.com/paper/2208.13686/) (2022)
- [Deep-Motion-Net: GNN-based volumetric organ shape reconstruction from single-view 2D projections](https://mlsanity.com/paper/2407.06692/) (2024)
- [Nonperiodic dynamic CT reconstruction using backward-warping INR with regularization of diffeomorphism (BIRD)](https://mlsanity.com/paper/2505.03463/) (2025)

## Abstract (arXiv)

This paper introduces a deep neural network based method, i.e., DeepOrganNet, to generate and visualize high-fidelity 3D / 4D organ geometric models from single-view medical image in real time. Traditional 3D / 4D medical image reconstruction requires near hundreds of projections, which cost insufferable computational time and deliver undesirable high imaging / radiation dose to human subjects. Moreover, it always needs further notorious processes to extract the accurate 3D organ models subsequently. To our knowledge, there is no method directly and explicitly reconstructing multiple 3D organ meshes from a single 2D medical grayscale image on the fly. Given single-view 2D medical images, e.g., 3D / 4D-CT projections or X-ray images, our end-to-end DeepOrganNet framework can efficiently and effectively reconstruct 3D / 4D lung models with a variety of geometric shapes by learning the smooth deformation fields from multiple templates based on a trivariate tensor-product deformation technique, leveraging an informative latent descriptor extracted from input 2D images. The proposed method can guarantee to generate high-quality and high-fidelity manifold meshes for 3D / 4D lung models. The major contributions of this work are to accurately reconstruct the 3D organ shapes from 2D single-view projection, significantly improve the procedure time to allow on-the-fly visualization, and dramatically reduce the imaging dose for human subjects. Experimental results are evaluated and compared with the traditional reconstruction method and the state-of-the-art in deep learning, by using extensive 3D and 4D examples from synthetic phantom and real patient datasets. The proposed method only needs several milliseconds to generate organ meshes with 10K vertices, which has a great potential to be used in real-time image guided radiation therapy (IGRT).