# A Generative Approach for Image Registration of Visible-Thermal (VT) Cancer Faces

arXiv: https://arxiv.org/abs/2308.12271 · HTML: https://mlsanity.com/paper/2308.12271/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 23 Aug 2023.

## Builds on (most cited first, 15 of 15)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [Breaking the Dilemma of Medical Image-to-image Translation](https://mlsanity.com/paper/2110.06465/) (2021)
- [Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation](https://mlsanity.com/paper/2003.08073/) (2020)
- [A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset](https://mlsanity.com/paper/2101.02637/) (2021)
- [A new face database simultaneously acquired in visible, near infrared and thermal spectrum](https://mlsanity.com/paper/2202.13864/) (2022)
- [The Use of AI for Thermal Emotion Recognition: A Review of Problems and Limitations in Standard Design and Data](https://mlsanity.com/paper/2009.10589/) (2020)
- [When Visible-to-Thermal Facial GAN Beats Conditional Diffusion](https://mlsanity.com/paper/2302.09395/) (2023)
- [Intelligent Sight and Sound: A Chronic Cancer Pain Dataset](https://mlsanity.com/paper/2204.04214/) (2022)
- [Vista-Morph: Unsupervised Image Registration of Visible-Thermal Facial Pairs](https://mlsanity.com/paper/2306.06505/) (2023)

## Abstract (arXiv)

Since thermal imagery offers a unique modality to investigate pain, the U.S. National Institutes of Health (NIH) has collected a large and diverse set of cancer patient facial thermograms for AI-based pain research. However, differing angles from camera capture between thermal and visible sensors has led to misalignment between Visible-Thermal (VT) images. We modernize the classic computer vision task of image registration by applying and modifying a generative alignment algorithm to register VT cancer faces, without the need for a reference or alignment parameters. By registering VT faces, we demonstrate that the quality of thermal images produced in the generative AI downstream task of Visible-to-Thermal (V2T) image translation significantly improves up to 52.5%, than without registration. Images in this paper have been approved by the NIH NCI for public dissemination.