# 3DPX: Single Panoramic X-ray Analysis Guided by 3D Oral Structure Reconstruction

arXiv: https://arxiv.org/abs/2409.18701 · HTML: https://mlsanity.com/paper/2409.18701/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 27 Sep 2024.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [PVT v2: Improved Baselines with Pyramid Vision Transformer](https://mlsanity.com/paper/2106.13797/) (2021)
- [Contrastive Learning of Medical Visual Representations from Paired Images and Text](https://mlsanity.com/paper/2010.00747/) (2020)
- [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022)
- [Bidirectional Projection Network for Cross Dimension Scene Understanding](https://mlsanity.com/paper/2103.14326/) (2021)
- [X2CT-GAN: Reconstructing CT from Biplanar X-Rays with Generative Adversarial Networks](https://mlsanity.com/paper/1905.06902/) (2019)
- [Multi-view PointNet for 3D Scene Understanding](https://mlsanity.com/paper/1909.13603/) (2019)
- [MedNeRF: Medical Neural Radiance Fields for Reconstructing 3D-aware CT-Projections from a Single X-ray](https://mlsanity.com/paper/2202.01020/) (2022)
- [Pri3D: Can 3D Priors Help 2D Representation Learning?](https://mlsanity.com/paper/2104.11225/) (2021)
- [3D-to-2D Distillation for Indoor Scene Parsing](https://mlsanity.com/paper/2104.02243/) (2021)
- [Training Deeper Convolutional Networks with Deep Supervision](https://mlsanity.com/paper/1505.02496/) (2015)
- [Deep Supervision with Intermediate Concepts](https://mlsanity.com/paper/1801.03399/) (2018)
- [Diffusion-Based Hierarchical Multi-Label Object Detection to Analyze Panoramic Dental X-rays](https://mlsanity.com/paper/2303.06500/) (2023)
- [X2Teeth: 3D Teeth Reconstruction from a Single Panoramic Radiograph](https://mlsanity.com/paper/2108.13004/) (2021)
- [3DPX: Progressive 2D-to-3D Oral Image Reconstruction with Hybrid MLP-CNN Networks](https://mlsanity.com/paper/2408.01292/) (2024)
- [Full-resolution MLPs Empower Medical Dense Prediction](https://mlsanity.com/paper/2311.16707/) (2023)

## Abstract (arXiv)

Panoramic X-ray (PX) is a prevalent modality in dentistry practice owing to its wide availability and low cost. However, as a 2D projection of a 3D structure, PX suffers from anatomical information loss and PX diagnosis is limited compared to that with 3D imaging modalities. 2D-to-3D reconstruction methods have been explored for the ability to synthesize the absent 3D anatomical information from 2D PX for use in PX image analysis. However, there are challenges in leveraging such 3D synthesized reconstructions. First, inferring 3D depth from 2D images remains a challenging task with limited accuracy. The second challenge is the joint analysis of 2D PX with its 3D synthesized counterpart, with the aim to maximize the 2D-3D synergy while minimizing the errors arising from the synthesized image. In this study, we propose a new method termed 3DPX - PX image analysis guided by 2D-to-3D reconstruction, to overcome these challenges. 3DPX consists of (i) a novel progressive reconstruction network to improve 2D-to-3D reconstruction and, (ii) a contrastive-guided bidirectional multimodality alignment module for 3D-guided 2D PX classification and segmentation tasks. The reconstruction network progressively reconstructs 3D images with knowledge imposed on the intermediate reconstructions at multiple pyramid levels and incorporates Multilayer Perceptrons to improve semantic understanding. The downstream networks leverage the reconstructed images as 3D anatomical guidance to the PX analysis through feature alignment, which increases the 2D-3D synergy with bidirectional feature projection and decease the impact of potential errors with contrastive guidance. Extensive experiments on two oral datasets involving 464 studies demonstrate that 3DPX outperforms the state-of-the-art methods in various tasks including 2D-to-3D reconstruction, PX classification and lesion segmentation.