# SAR-to-Optical Image Translation via Thermodynamics-inspired Network

arXiv: https://arxiv.org/abs/2305.13839 · HTML: https://mlsanity.com/paper/2305.13839/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 53 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias](https://mlsanity.com/paper/2106.03348/) (2021)
- [Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations](https://mlsanity.com/paper/1710.10121/) (2017)
- [MirrorGAN: Learning Text-to-image Generation by Redescription](https://mlsanity.com/paper/1903.05854/) (2019)
- [Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things](https://mlsanity.com/paper/2011.08612/) (2020)
- [AttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Networks](https://mlsanity.com/paper/1911.11897/) (2019)
- [Dual Contrastive Learning for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2104.07689/) (2021)
- [The SEN1-2 Dataset for Deep Learning in SAR-Optical Data Fusion](https://mlsanity.com/paper/1807.01569/) (2018)
- [Nighttime Dehazing with a Synthetic Benchmark](https://mlsanity.com/paper/2008.03864/) (2020)
- [PolyNet: A Pursuit of Structural Diversity in Very Deep Networks](https://mlsanity.com/paper/1611.05725/) (2016)
- [Adversarial Bipartite Graph Learning for Video Domain Adaptation](https://mlsanity.com/paper/2007.15829/) (2020)
- [Multi-Channel Attention Selection GANs for Guided Image-to-Image Translation](https://mlsanity.com/paper/2002.01048/) (2020)
- [FAMED-Net: A Fast and Accurate Multi-scale End-to-end Dehazing Network](https://mlsanity.com/paper/1906.04334/) (2019)
- [Reciprocal Translation between SAR and Optical Remote Sensing Images with Cascaded-Residual Adversarial Networks](https://mlsanity.com/paper/1901.08236/) (2019)
- [Generating High Quality Visible Images from SAR Images Using CNNs](https://mlsanity.com/paper/1802.10036/) (2018)
- [Fully Point-wise Convolutional Neural Network for Modeling Statistical Regularities in Natural Images](https://mlsanity.com/paper/1801.06302/) (2018)
- [Retrieval Guided Unsupervised Multi-domain Image-to-Image Translation](https://mlsanity.com/paper/2008.04991/) (2020)

## Built on by (most cited first, 1 of 1)

- [Multi-Sensor Diffusion-Driven Optical Image Translation for Large-Scale Applications](https://mlsanity.com/paper/2404.11243/) (2024)

## Abstract (arXiv)

Synthetic aperture radar (SAR) is prevalent in the remote sensing field but is difficult to interpret in human visual perception. Recently, SAR-to-optical (S2O) image conversion methods have provided a prospective solution for interpretation. However, since there is a huge domain difference between optical and SAR images, they suffer from low image quality and geometric distortion in the produced optical images. Motivated by the analogy between pixels during the S2O image translation and molecules in a heat field, Thermodynamics-inspired Network for SAR-to-Optical Image Translation (S2O-TDN) is proposed in this paper. Specifically, we design a Third-order Finite Difference (TFD) residual structure in light of the TFD equation of thermodynamics, which allows us to efficiently extract inter-domain invariant features and facilitate the learning of the nonlinear translation mapping. In addition, we exploit the first law of thermodynamics (FLT) to devise an FLT-guided branch that promotes the state transition of the feature values from the unstable diffusion state to the stable one, aiming to regularize the feature diffusion and preserve image structures during S2O image translation. S2O-TDN follows an explicit design principle derived from thermodynamic theory and enjoys the advantage of explainability. Experiments on the public SEN1-2 dataset show the advantages of the proposed S2O-TDN over the current methods with more delicate textures and higher quantitative results.