Towards Highly Expressive Machine Learning Models of Non-Melanoma Skin Cancer
Not on the map. Why?
Abstract (arXiv)
Pathologists have a rich vocabulary with which they can describe all the nuances of cellular morphology. In their world, there is a natural pairing of images and words. Recent advances demonstrate that machine learning models can now be trained to learn high-quality image features and represent them as discrete units of information. This enables natural language, which is also discrete, to be jointly modelled alongside the imaging, resulting in a description of the contents of the imaging. Here we present experiments in applying discrete modelling techniques to the problem domain of non-melanoma skin cancer, specifically, histological images of Intraepidermal Carcinoma (IEC). Implementing a VQ-GAN model to reconstruct high-resolution (256x256) images of IEC images, we trained a sequence-to-sequence transformer to generate natural language descriptions using pathologist terminology. Combined with the idea of interactive concept vectors available by using continuous generative methods, we demonstrate an additional angle of interpretability. The result is a promising means of working towards highly expressive machine learning systems which are not only useful as predictive/classification tools, but also means to further our scientific understanding of disease.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 42 references that are in the map. First posted on arXiv 9 Jul 2022.
Builds on
24 of 42 references.
References · 24
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2017Neural Discrete Representation Learningcited by 4,650
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2021Zero-Shot Text-to-Image Generationcited by 4,276
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
12 more
- 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
- 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
- 2016Generating Images with Perceptual Similarity Metrics based on Deep Networkscited by 221
- 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
- 2020Adversarial Latent Autoencoderscited by 155
- 2017MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Networkcited by 56
- 2019Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networkscited by 43
- 2019PathologyGAN: Learning deep representations of cancer tissuecited by 32
- 2019Holistic and Comprehensive Annotation of Clinically Significant Findings on Diverse CT Images: Learning from Radiology Reports and Label Ontologycited by 18
- 2018Pioneer Networks: Progressively Growing Generative Autoencodercited by 4
- 2020Learning a low dimensional manifold of real cancer tissue with PathologyGANcited by 3
- 2020Predicting Prostate Cancer-Specific Mortality with A.I.-based Gleason Gradingcited by 3
Built on by
Citing papers per year: 2022: 1.
1 paper.