# End-To-End Convolutional Neural Network for 3D Reconstruction of Knee Bones From Bi-Planar X-Ray Images

arXiv: https://arxiv.org/abs/2004.00871 · HTML: https://mlsanity.com/paper/2004.00871/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2020.

## Built on by (most cited first, 28 of 28)

- [NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction](https://mlsanity.com/paper/2209.14540/) (2022)
- [Structure-Aware Sparse-View X-ray 3D Reconstruction](https://mlsanity.com/paper/2311.10959/) (2023)
- [Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays](https://mlsanity.com/paper/2303.05297/) (2023)
- [Geometry-Aware Attenuation Learning for Sparse-View CBCT Reconstruction](https://mlsanity.com/paper/2303.14739/) (2023)
- [DiffuX2CT: Diffusion Learning to Reconstruct CT Images from Biplanar X-Rays](https://mlsanity.com/paper/2407.13545/) (2024)
- [DIFR3CT: Latent Diffusion for Probabilistic 3D CT Reconstruction from Few Planar X-Rays](https://mlsanity.com/paper/2408.15118/) (2024)
- [IGCN: Image-to-graph Convolutional Network for 2D/3D Deformable Registration](https://mlsanity.com/paper/2111.00484/) (2021)
- [DeepCA: Deep Learning-based 3D Coronary Artery Tree Reconstruction from Two 2D Non-simultaneous X-ray Angiography Projections](https://mlsanity.com/paper/2407.14616/) (2024)
- [3D Vessel Reconstruction from Sparse-View Dynamic DSA Images via Vessel Probability Guided Attenuation Learning](https://mlsanity.com/paper/2405.10705/) (2024)
- [Improving Computed Tomography (CT) Reconstruction via 3D Shape Induction](https://mlsanity.com/paper/2208.10937/) (2022)
- [Oral-3Dv2: 3D Oral Reconstruction from Panoramic X-Ray Imaging with Implicit Neural Representation](https://mlsanity.com/paper/2303.12123/) (2023)
- [NeBLa: Neural Beer-Lambert for 3D Reconstruction of Oral Structures from Panoramic Radiographs](https://mlsanity.com/paper/2304.04027/) (2023)
- [NeAS: 3D Reconstruction from X-ray Images using Neural Attenuation Surface](https://mlsanity.com/paper/2503.07491/) (2025)
- [Transfer Learning from an Artificial Radiograph-landmark Dataset for Registration of the Anatomic Skull Model to Dual Fluoroscopic X-ray Images](https://mlsanity.com/paper/2108.06466/) (2021)
- [Domain adaptation strategies for 3D reconstruction of the lumbar spine using real fluoroscopy data](https://mlsanity.com/paper/2401.16027/) (2024)
- [3DDX: Bone Surface Reconstruction from a Single Standard-Geometry Radiograph via Dual-Face Depth Estimation](https://mlsanity.com/paper/2409.16702/) (2024)
- [MRIS: A Multi-modal Retrieval Approach for Image Synthesis on Diverse Modalities](https://mlsanity.com/paper/2303.10249/) (2023)
- [Multi-view X-ray Image Synthesis with Multiple Domain Disentanglement from CT Scans](https://mlsanity.com/paper/2404.11889/) (2024)
- [X-Recon: Learning-based Patient-specific High-Resolution CT Reconstruction from Orthogonal X-Ray Images](https://mlsanity.com/paper/2407.15356/) (2024)
- [Self-Supervised Multiview Xray Matching](https://mlsanity.com/paper/2507.00287/) (2025)
- [Real-Time Reconstruction of 3D Bone Models via Very-Low-Dose Protocols](https://mlsanity.com/paper/2508.13947/) (2025)
- [Biomedical Image Reconstruction: A Survey](https://mlsanity.com/paper/2301.11813/) (2023)
- [The Impact of Loss Functions and Scene Representations for 3D/2D Registration on Single-view Fluoroscopic X-ray Pose Estimation](https://mlsanity.com/paper/2308.00214/) (2023)
- [Benchmarking Encoder-Decoder Architectures for Biplanar X-ray to 3D Shape Reconstruction](https://mlsanity.com/paper/2309.13587/) (2023)
- [EAR: Edge-Aware Reconstruction of 3-D vertebrae structures from bi-planar X-ray images](https://mlsanity.com/paper/2407.20937/) (2024)
- [Reconstruct Spine CT from Biplanar X-Rays via Diffusion Learning](https://mlsanity.com/paper/2408.09731/) (2024)
- [Coarse-Fine View Attention Alignment-Based GAN for CT Reconstruction from Biplanar X-Rays](https://mlsanity.com/paper/2408.09736/) (2024)
- [IXGS-Intraoperative 3D Reconstruction from Sparse, Arbitrarily Posed Real X-rays](https://mlsanity.com/paper/2504.14699/) (2025)

## Abstract (arXiv)

We present an end-to-end Convolutional Neural Network (CNN) approach for 3D reconstruction of knee bones directly from two bi-planar X-ray images. Clinically, capturing the 3D models of the bones is crucial for surgical planning, implant fitting, and postoperative evaluation. X-ray imaging significantly reduces the exposure of patients to ionizing radiation compared to Computer Tomography (CT) imaging, and is much more common and inexpensive compared to Magnetic Resonance Imaging (MRI) scanners. However, retrieving 3D models from such 2D scans is extremely challenging. In contrast to the common approach of statistically modeling the shape of each bone, our deep network learns the distribution of the bones' shapes directly from the training images. We train our model with both supervised and unsupervised losses using Digitally Reconstructed Radiograph (DRR) images generated from CT scans. To apply our model to X-Ray data, we use style transfer to transform between X-Ray and DRR modalities. As a result, at test time, without further optimization, our solution directly outputs a 3D reconstruction from a pair of bi-planar X-ray images, while preserving geometric constraints. Our results indicate that our deep learning model is very efficient, generalizes well and produces high quality reconstructions.