# T-FAKE: Synthesizing Thermal Images for Facial Landmarking

arXiv: https://arxiv.org/abs/2408.15127 · HTML: https://mlsanity.com/paper/2408.15127/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 72 references that are in the map.
- First posted on arXiv 27 Aug 2024.

## Builds on (most cited first, 30 of 33)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [The MegaFace Benchmark: 1 Million Faces for Recognition at Scale](https://mlsanity.com/paper/1512.00596/) (2015)
- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [Sensitivity and Generalization in Neural Networks: an Empirical Study](https://mlsanity.com/paper/1802.08760/) (2018)
- [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018)
- [LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood](https://mlsanity.com/paper/2004.02980/) (2020)
- [3D face reconstruction with dense landmarks](https://mlsanity.com/paper/2204.02776/) (2022)
- [Deep Alignment Network: A convolutional neural network for robust face alignment](https://mlsanity.com/paper/1706.01789/) (2017)
- [STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection](https://mlsanity.com/paper/2306.02763/) (2023)
- [QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translation](https://mlsanity.com/paper/2203.08483/) (2022)
- [ADNet: Leveraging Error-Bias Towards Normal Direction in Face Alignment](https://mlsanity.com/paper/2109.05721/) (2021)
- [3FabRec: Fast Few-shot Face alignment by Reconstruction](https://mlsanity.com/paper/1911.10448/) (2019)
- [Attention Mesh: High-fidelity Face Mesh Prediction in Real-time](https://mlsanity.com/paper/2006.10962/) (2020)
- [Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks](https://mlsanity.com/paper/1812.05155/) (2018)
- [A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset](https://mlsanity.com/paper/2101.02637/) (2021)
- [PatchNR: Learning from Very Few Images by Patch Normalizing Flow Regularization](https://mlsanity.com/paper/2205.12021/) (2022)
- [T2V-DDPM: Thermal to Visible Face Translation using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2209.08814/) (2022)
- [Wasserstein Patch Prior for Image Superresolution](https://mlsanity.com/paper/2109.12880/) (2021)
- [TV-GAN: Generative Adversarial Network Based Thermal to Visible Face Recognition](https://mlsanity.com/paper/1712.02514/) (2017)
- [WPPNets and WPPFlows: The Power of Wasserstein Patch Priors for Superresolution](https://mlsanity.com/paper/2201.08157/) (2022)
- [Thermal to Visible Synthesis of Face Images using Multiple Regions](https://mlsanity.com/paper/1803.07599/) (2018)
- [Learning from small data sets: Patch-based regularizers in inverse problems for image reconstruction](https://mlsanity.com/paper/2312.16611/) (2023)
- [Sejong Face Database: A Multi-Modal Disguise Face Database](https://mlsanity.com/paper/2106.07186/) (2021)

## Built on by (most cited first, 7 of 7)

- [Slicing Wasserstein Over Wasserstein Via Functional Optimal Transport](https://mlsanity.com/paper/2509.22138/) (2025)
- [Site-Level Fine-Tuning with Progressive Layer Freezing: Towards Robust Prediction of Bronchopulmonary Dysplasia from Day-1 Chest Radiographs in Extremely Preterm Infants](https://mlsanity.com/paper/2507.12269/) (2025)
- [Lightweight Facial Landmark Detection in Thermal Images via Multi-Level Cross-Modal Knowledge Transfer](https://mlsanity.com/paper/2510.11128/) (2025)
- [HOT-POT: Optimal Transport for Sparse Stereo Matching](https://mlsanity.com/paper/2601.12423/) (2026)
- [Training with synthetic data for drone detection in thermal imagery](https://mlsanity.com/paper/2608.17799/) (2026)
- [FVeinSyn: Synthetic Finger Vein Image Generator](https://mlsanity.com/paper/2608.27527/) (2026)
- [SynThermFace: Amplifying Limited Paired Data for Visible-Thermal Face Recognition via Synthetic Data Generation](https://mlsanity.com/paper/2609.10303/) (2026)

## Abstract (arXiv)

Facial analysis is a key component in a wide range of applications such as healthcare, autonomous driving, and entertainment. Despite the availability of various facial RGB datasets, the thermal modality, which plays a crucial role in life sciences, medicine, and biometrics, has been largely overlooked. To address this gap, we introduce the T-FAKE dataset, a new large-scale synthetic thermal dataset with sparse and dense landmarks. To facilitate the creation of the dataset, we propose a novel RGB2Thermal loss function, which enables the domain-adaptive transfer of RGB faces to thermal style. By utilizing the Wasserstein distance between thermal and RGB patches and the statistical analysis of clinical temperature distributions on faces, we ensure that the generated thermal images closely resemble real samples. Using RGB2Thermal style transfer based on our RGB2Thermal loss function, we create the large-scale synthetic thermal T-FAKE dataset with landmark and segmentation annotations. Leveraging our novel T-FAKE dataset, probabilistic landmark prediction, and label adaptation networks, we demonstrate significant improvements in landmark detection methods on thermal images across different landmark conventions. Our models show excellent performance with both sparse 70-point landmarks and dense 478-point landmark annotations. Moreover, our RGB2Thermal loss leads to notable results in terms of perceptual evaluation and temperature prediction.