Enhanced Masked Image Modeling for Analysis of Dental Panoramic Radiographs
On the map: Knowledge distillation → Distillation for medical imaging
Abstract (arXiv)
The computer-assisted radiologic informative report has received increasing research attention to facilitate diagnosis and treatment planning for dental care providers. However, manual interpretation of dental images is limited, expensive, and time-consuming. Another barrier in dental imaging is the limited number of available images for training, which is a challenge in the era of deep learning. This study proposes a novel self-distillation (SD) enhanced self-supervised learning on top of the masked image modeling (SimMIM) Transformer, called SD-SimMIM, to improve the outcome with a limited number of dental radiographs. In addition to the prediction loss on masked patches, SD-SimMIM computes the self-distillation loss on the visible patches. We apply SD-SimMIM on dental panoramic X-rays for teeth numbering, detection of dental restorations and orthodontic appliances, and instance segmentation tasks. Our results show that SD-SimMIM outperforms other self-supervised learning methods. Furthermore, we augment and improve the annotation of an existing dataset of panoramic X-rays.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation. Builds on 10 of its 19 references that are in the map. First posted on arXiv 18 Jun 2023.
Builds on
10 of 19 references.
References · 10
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
- 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
- 2022Uniform Masking: Enabling MAE Pre-training for Pyramid-based Vision Transformers with Localitycited by 57
- 2022Self-Supervised Learning with Masked Image Modeling for Teeth Numbering, Detection of Dental Restorations, and Instance Segmentation in Dental Panoramic Radiographscited by 8
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Towards Better Dental AI: A Multimodal Benchmark and Instruction Dataset for Panoramic X-ray Analysis
- 2023Diffusion-Based Hierarchical Multi-Label Object Detection to Analyze Panoramic Dental X-rays
- 2026Weighted Knowledge Distillation for Semi-Supervised Segmentation of Maxillary Sinus in Panoramic X-ray Images
- 2026OralGPT-Plus: Learning to Use Visual Tools via Reinforcement Learning for Panoramic X-ray Analysis
- 2025When CNNs Outperform Transformers and Mambas: Revisiting Deep Architectures for Dental Caries Segmentation
- 2023DENTEX: Dental Enumeration and Tooth Pathosis Detection Benchmark for Panoramic X-ray
- 2024Detection Transformer for Teeth Detection, Segmentation, and Numbering in Oral Rare Diseases: Focus on Data Augmentation and Inpainting Techniques
- 2026AgenTeeth: A Model-Agnostic Framework for Suppressing Hallucination in Frozen Vision-Language Models on Dental X-Rays via Tool Evidence Injection