OrthoGAN:High-Precision Image Generation for Teeth Orthodontic Visualization
Not on the map. Why?
Abstract (arXiv)
Patients take care of what their teeth will be like after the orthodontics. Orthodontists usually describe the expectation movement based on the original smile images, which is unconvincing. The growth of deep-learning generative models change this situation. It can visualize the outcome of orthodontic treatment and help patients foresee their future teeth and facial appearance. While previous studies mainly focus on 2D or 3D virtual treatment outcome (VTO) at a profile level, the problem of simulating treatment outcome at a frontal facial image is poorly explored. In this paper, we build an efficient and accurate system for simulating virtual teeth alignment effects in a frontal facial image. Our system takes a frontal face image of a patient with visible malpositioned teeth and the patient's 3D scanned teeth model as input, and progressively generates the visual results of the patient's teeth given the specific orthodontics planning steps from the doctor (i.e., the specification of translations and rotations of individual tooth). We design a multi-modal encoder-decoder based generative model to synthesize identity-preserving frontal facial images with aligned teeth. In addition, the original image color information is used to optimize the orthodontic outcomes, making the results more natural. We conduct extensive qualitative and clinical experiments and also a pilot study to validate our method.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 55 of its 99 references that are in the map. First posted on arXiv 29 Dec 2022.
Builds on
55 of 99 references.
References · 55
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2022Generative Adversarial Networkscited by 5,603
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
- 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
38 more
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
- 2022Video Diffusion Modelscited by 2,280
- 2014Conditional Generative Adversarial Netscited by 2,152
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
- 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
- 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
- 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,518
- 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
- 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
- 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
- 2016An overview of gradient descent optimization algorithmscited by 1,021
- 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
- 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
- 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
- 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
- 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
- 2015Holistically-Nested Edge Detectioncited by 590
- 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
- 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
- 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
- 2022NoPe-NeRF: Optimising Neural Radiance Field with No Pose Priorcited by 235
- 2016Unrolled Generative Adversarial Networkscited by 208
- 2018Multimodal Generative Models for Scalable Weakly-Supervised Learningcited by 200
- 2021High-Fidelity GAN Inversion for Image Attribute Editingcited by 197
- 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
- 2021EditGAN: High-Precision Semantic Image Editingcited by 156
- 2022SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Videocited by 144
- 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
- 2020Differentiable Rendering: A Surveycited by 123
- 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
- 2017MarrNet: 3D Shape Reconstruction via 2.5D Sketchescited by 108
- 2018SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesiscited by 81
- 2021HairCLIP: Design Your Hair by Text and Reference Imagecited by 79
- 2021Multimodal Conditional Image Synthesis with Product-of-Experts GANscited by 75
- 2019RelGAN: Multi-Domain Image-to-Image Translation via Relative Attributescited by 56
- 2019PI-RCNN: An Efficient Multi-sensor 3D Object Detector with Point-based Attentive Cont-conv Fusion Modulecited by 56
- 2017Generative Models of Visually Grounded Imaginationcited by 44
50 most cited of 55 All 55 (JSON)
Built on by
Not cited yet.