# Robust Self-Supervised Cross-Modal Super-Resolution against Real-World Misaligned Observations

arXiv: https://arxiv.org/abs/2602.18822 · HTML: https://mlsanity.com/paper/2602.18822/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 49 of its 98 references that are in the map.
- First posted on arXiv 21 Feb 2026.

## Builds on (most cited first, 30 of 49)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [Non-Local Recurrent Network for Image Restoration](https://mlsanity.com/paper/1806.02919/) (2018)
- [DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion](https://mlsanity.com/paper/2303.06840/) (2023)
- [Equivariant Multi-Modality Image Fusion](https://mlsanity.com/paper/2305.11443/) (2023)
- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [Discrete Cosine Transform Network for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2104.06977/) (2021)
- [Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation](https://mlsanity.com/paper/2003.08073/) (2020)
- [Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark Dataset and Baseline](https://mlsanity.com/paper/2104.06174/) (2021)
- [Deformable Kernel Networks for Joint Image Filtering](https://mlsanity.com/paper/1910.08373/) (2019)
- [Fast End-to-End Trainable Guided Filter](https://mlsanity.com/paper/1803.05619/) (2018)
- [MINIMA: Modality Invariant Image Matching](https://mlsanity.com/paper/2412.19412/) (2024)
- [Guided Depth Super-Resolution by Deep Anisotropic Diffusion](https://mlsanity.com/paper/2211.11592/) (2022)
- [Image Demoireing with Learnable Bandpass Filters](https://mlsanity.com/paper/2004.00406/) (2020)
- [Tri-Perspective View Decomposition for Geometry-Aware Depth Completion](https://mlsanity.com/paper/2403.15008/) (2024)
- [Deep Lucas-Kanade Homography for Multimodal Image Alignment](https://mlsanity.com/paper/2104.11693/) (2021)
- [Spherical Space Feature Decomposition for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2303.08942/) (2023)
- [SGNet: Structure Guided Network via Gradient-Frequency Awareness for Depth Map Super-Resolution](https://mlsanity.com/paper/2312.05799/) (2023)
- [Learnable Tree Filter for Structure-preserving Feature Transform](https://mlsanity.com/paper/1909.12513/) (2019)
- [Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening](https://mlsanity.com/paper/2404.07543/) (2024)
- [Unaligned RGB Guided Hyperspectral Image Super-Resolution with Spatial-Spectral Concordance](https://mlsanity.com/paper/2505.02109/) (2025)
- [Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration](https://mlsanity.com/paper/2402.18933/) (2024)
- [Multi-Modal MRI Reconstruction Assisted with Spatial Alignment Network](https://mlsanity.com/paper/2108.05603/) (2021)
- [Deep End-to-End Alignment and Refinement for Time-of-Flight RGB-D Module](https://mlsanity.com/paper/1909.07623/) (2019)
- [Structure Flow-Guided Network for Real Depth Super-Resolution](https://mlsanity.com/paper/2301.13416/) (2023)
- [DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution](https://mlsanity.com/paper/2410.11666/) (2024)
- [Hyperspectral Image Super Resolution with Real Unaligned RGB Guidance](https://mlsanity.com/paper/2302.06298/) (2023)
- [Learning Mutual Modulation for Self-Supervised Cross-Modal Super-Resolution](https://mlsanity.com/paper/2207.09156/) (2022)

## Abstract (arXiv)

Cross-modal super-resolution (SR) on real-world misaligned data is challenging, as only unlabeled low-resolution (LR) source and high-resolution (HR) guide images with complex spatial misalignment are available. Previous methods either rely on simulated training data or adopt suboptimal alignment strategies that overlook cross-modal dependencies, limiting their practical performance. To address these issues, we propose RobSelf, a self-supervised model that jointly optimizes a misalignment-aware feature translator and a content-aware reference filter online. The translator resolves unsupervised cross-modal and cross-resolution alignment via weakly-supervised, misalignment-aware translation, yielding an aligned guide feature. Guided by this feature, the filter performs reference-based discriminative self-enhancement on the source, enabling SR prediction with high resolution and high fidelity. Experiments on synthesized data and collected real-world data demonstrate that RobSelf achieves state-of-the-art performance, outperforming existing self-supervised and supervised methods. Moreover, it achieves superior efficiency, being up to 15.3× faster than prior self-supervised methods.