# Structure-Aware Sparse-View X-ray 3D Reconstruction

arXiv: https://arxiv.org/abs/2311.10959 · HTML: https://mlsanity.com/paper/2311.10959/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 60 references that are in the map.
- First posted on arXiv 18 Nov 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion](https://mlsanity.com/paper/2212.06135/) (2022)
- [Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement](https://mlsanity.com/paper/2303.06705/) (2023)
- [TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up](https://mlsanity.com/paper/2102.07074/) (2021)
- [DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation](https://mlsanity.com/paper/1506.06448/) (2015)
- [StyleSwin: Transformer-based GAN for High-resolution Image Generation](https://mlsanity.com/paper/2112.10762/) (2021)
- [Solving 3D Inverse Problems using Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2211.10655/) (2022)
- [Hallucinated Neural Radiance Fields in the Wild](https://mlsanity.com/paper/2111.15246/) (2021)
- [Generative Adversarial Transformers](https://mlsanity.com/paper/2103.01209/) (2021)
- [Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2111.07910/) (2021)
- [Light Field Neural Rendering](https://mlsanity.com/paper/2112.09687/) (2021)
- [NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction](https://mlsanity.com/paper/2209.14540/) (2022)
- [X2CT-GAN: Reconstructing CT from Biplanar X-Rays with Generative Adversarial Networks](https://mlsanity.com/paper/1905.06902/) (2019)
- [MedNeRF: Medical Neural Radiance Fields for Reconstructing 3D-aware CT-Projections from a Single X-ray](https://mlsanity.com/paper/2202.01020/) (2022)
- [Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging](https://mlsanity.com/paper/2205.10102/) (2022)
- [NeuRBF: A Neural Fields Representation with Adaptive Radial Basis Functions](https://mlsanity.com/paper/2309.15426/) (2023)

## Built on by (most cited first, 30 of 72)

- [A Survey on 3D Gaussian Splatting](https://mlsanity.com/paper/2401.03890/) (2024)
- [PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation](https://mlsanity.com/paper/2404.13026/) (2024)
- [R2-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction](https://mlsanity.com/paper/2405.20693/) (2024)
- [Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis](https://mlsanity.com/paper/2403.04116/) (2024)
- [Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Review](https://mlsanity.com/paper/2405.03417/) (2024)
- [HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting](https://mlsanity.com/paper/2405.15125/) (2024)
- [NTIRE 2024 Challenge on Low Light Image Enhancement: Methods and Results](https://mlsanity.com/paper/2404.14248/) (2024)
- [LangSurf: Language-Embedded Surface Gaussians for 3D Scene Understanding](https://mlsanity.com/paper/2412.17635/) (2024)
- [Volumetrically Consistent 3D Gaussian Rasterization](https://mlsanity.com/paper/2412.03378/) (2024)
- [MASIV: Toward Material-Agnostic System Identification from Videos](https://mlsanity.com/paper/2508.01112/) (2025)
- [Neural Radiance Fields in Medical Imaging: A Survey](https://mlsanity.com/paper/2402.17797/) (2024)
- [Discretized Gaussian Representation for Tomographic Reconstruction](https://mlsanity.com/paper/2411.04844/) (2024)
- [X-LRM: X-ray Large Reconstruction Model for Extremely Sparse-View Computed Tomography Recovery in One Second](https://mlsanity.com/paper/2503.06382/) (2025)
- [X2-Gaussian: 4D Radiative Gaussian Splatting for Continuous-time Tomographic Reconstruction](https://mlsanity.com/paper/2503.21779/) (2025)
- [X-GRM: Large Gaussian Reconstruction Model for Sparse-view X-rays to Computed Tomography](https://mlsanity.com/paper/2505.15235/) (2025)
- [Are Pixel-Wise Metrics Reliable for Sparse-View Computed Tomography Reconstruction?](https://mlsanity.com/paper/2506.02093/) (2025)
- [3D Vessel Reconstruction from Sparse-View Dynamic DSA Images via Vessel Probability Guided Attenuation Learning](https://mlsanity.com/paper/2405.10705/) (2024)
- [GR-Gaussian: Graph-Based Radiative Gaussian Splatting for Sparse-View CT Reconstruction](https://mlsanity.com/paper/2508.02408/) (2025)
- [NeRF-CA: Dynamic Reconstruction of X-ray Coronary Angiography with Extremely Sparse-views](https://mlsanity.com/paper/2408.16355/) (2024)
- [Unsupervised Multi-Parameter Inverse Solving for Reducing Ring Artifacts in 3D X-Ray CBCT](https://mlsanity.com/paper/2412.05853/) (2024)
- [4DRGS: 4D Radiative Gaussian Splatting for Efficient 3D Vessel Reconstruction from Sparse-View Dynamic DSA Images](https://mlsanity.com/paper/2412.12919/) (2024)
- [Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering](https://mlsanity.com/paper/2505.17338/) (2025)
- [Differentiable Voxel-based X-ray Rendering Improves Sparse-View 3D CBCT Reconstruction](https://mlsanity.com/paper/2411.19224/) (2024)
- [NeAS: 3D Reconstruction from X-ray Images using Neural Attenuation Surface](https://mlsanity.com/paper/2503.07491/) (2025)
- [X-Field: A Physically Grounded Representation for 3D X-ray Reconstruction](https://mlsanity.com/paper/2503.08596/) (2025)
- [InnerGS: Internal Scenes Reconstruction and Segmentation via Factorized 3D Gaussian Splatting](https://mlsanity.com/paper/2508.13287/) (2025)
- [MedGS: Gaussian Splatting for Multi-Modal 3D Medical Imaging](https://mlsanity.com/paper/2509.16806/) (2025)
- [See More, Change Less: Anatomy-Aware Diffusion for Contrast Enhancement](https://mlsanity.com/paper/2512.07251/) (2025)
- [NeRF-based CBCT Reconstruction needs Normalization and Initialization](https://mlsanity.com/paper/2506.19742/) (2025)
- [Radioactive 3D Gaussian Ray Tracing for Tomographic Reconstruction](https://mlsanity.com/paper/2602.01057/) (2026)

## Abstract (arXiv)

X-ray, known for its ability to reveal internal structures of objects, is expected to provide richer information for 3D reconstruction than visible light. Yet, existing neural radiance fields (NeRF) algorithms overlook this important nature of X-ray, leading to their limitations in capturing structural contents of imaged objects. In this paper, we propose a framework, Structure-Aware X-ray Neural Radiodensity Fields (SAX-NeRF), for sparse-view X-ray 3D reconstruction. Firstly, we design a Line Segment-based Transformer (Lineformer) as the backbone of SAX-NeRF. Linefomer captures internal structures of objects in 3D space by modeling the dependencies within each line segment of an X-ray. Secondly, we present a Masked Local-Global (MLG) ray sampling strategy to extract contextual and geometric information in 2D projection. Plus, we collect a larger-scale dataset X3D covering wider X-ray applications. Experiments on X3D show that SAX-NeRF surpasses previous NeRF-based methods by 12.56 and 2.49 dB on novel view synthesis and CT reconstruction. Code, models, and data are released at https://github.com/caiyuanhao1998/SAX-NeRF