# Latent Harmony: Synergistic Unified UHD Image Restoration via Latent Space Regularization and Controllable Refinement

arXiv: https://arxiv.org/abs/2510.07961 · HTML: https://mlsanity.com/paper/2510.07961/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 52 references that are in the map.
- First posted on arXiv 9 Oct 2025.

## Builds on (most cited first, 30 of 35)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models](https://mlsanity.com/paper/2501.01423/) (2025)
- [MoE-LLaVA: Mixture of Experts for Large Vision-Language Models](https://mlsanity.com/paper/2401.15947/) (2024)
- [REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers](https://mlsanity.com/paper/2504.10483/) (2025)
- [AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation](https://mlsanity.com/paper/2403.14614/) (2024)
- [EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling](https://mlsanity.com/paper/2502.09509/) (2025)
- [Embedding Fourier for Ultra-High-Definition Low-Light Image Enhancement](https://mlsanity.com/paper/2302.11831/) (2023)
- [Improving the Diffusability of Autoencoders](https://mlsanity.com/paper/2502.14831/) (2025)
- [ProRes: Exploring Degradation-aware Visual Prompt for Universal Image Restoration](https://mlsanity.com/paper/2306.13653/) (2023)
- [Complexity Experts are Task-Discriminative Learners for Any Image Restoration](https://mlsanity.com/paper/2411.18466/) (2024)
- [Selective Hourglass Mapping for Universal Image Restoration Based on Diffusion Model](https://mlsanity.com/paper/2403.11157/) (2024)
- [Restoring Images in Adverse Weather Conditions via Histogram Transformer](https://mlsanity.com/paper/2407.10172/) (2024)
- [GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions](https://mlsanity.com/paper/2305.17863/) (2023)
- [Towards Effective Multiple-in-One Image Restoration: A Sequential and Prompt Learning Strategy](https://mlsanity.com/paper/2401.03379/) (2024)
- [Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Tasks](https://mlsanity.com/paper/2403.00644/) (2024)
- [Neural Degradation Representation Learning for All-In-One Image Restoration](https://mlsanity.com/paper/2310.12848/) (2023)
- [Prompt-In-Prompt Learning for Universal Image Restoration](https://mlsanity.com/paper/2312.05038/) (2023)
- [Correlation Matching Transformation Transformers for UHD Image Restoration](https://mlsanity.com/paper/2406.00629/) (2024)
- [LiteVAE: Lightweight and Efficient Variational Autoencoders for Latent Diffusion Models](https://mlsanity.com/paper/2405.14477/) (2024)
- [Catch Missing Details: Image Reconstruction with Frequency Augmented Variational Autoencoder](https://mlsanity.com/paper/2305.02541/) (2023)
- [Multi-Expert Adaptive Selection: Task-Balancing for All-in-One Image Restoration](https://mlsanity.com/paper/2407.19139/) (2024)
- [Epsilon-VAE: Denoising as Visual Decoding](https://mlsanity.com/paper/2410.04081/) (2024)
- [Alias-Free Latent Diffusion Models: Improving Fractional Shift Equivariance of Diffusion Latent Space](https://mlsanity.com/paper/2503.09419/) (2025)
- [Single UHD Image Dehazing via Interpretable Pyramid Network](https://mlsanity.com/paper/2202.08589/) (2022)
- [HAIR: Hypernetworks-based All-in-One Image Restoration](https://mlsanity.com/paper/2408.08091/) (2024)
- [Unleashing the Potential of the Semantic Latent Space in Diffusion Models for Image Dehazing](https://mlsanity.com/paper/2509.20091/) (2025)

## Built on by (most cited first, 3 of 3)

- [FinPercep-RM: A Fine-grained Reward Model and Co-evolutionary Curriculum for RL-based Real-world Super-Resolution](https://mlsanity.com/paper/2512.22647/) (2025)
- [FlowInOne:Unifying Multimodal Generation as Image-in, Image-out Flow Matching](https://mlsanity.com/paper/2604.06757/) (2026)
- [On the Global Photometric Alignment for Low-Level Vision](https://mlsanity.com/paper/2604.08172/) (2026)

## Abstract (arXiv)

Ultra-High Definition (UHD) image restoration faces a trade-off between computational efficiency and high-frequency detail retention. While Variational Autoencoders (VAEs) improve efficiency via latent-space processing, their Gaussian constraint often discards degradation-specific high-frequency information, hurting reconstruction fidelity. To overcome this, we propose Latent Harmony, a two-stage framework that redefines VAEs for UHD restoration by jointly regularizing the latent space and enforcing high-frequency-aware reconstruction.In Stage One, we introduce LH-VAE, which enhances semantic robustness through visual semantic constraints and progressive degradation perturbations, while latent equivariance strengthens high-frequency reconstruction.Stage Two jointly trains this refined VAE with a restoration model using High-Frequency Low-Rank Adaptation (HF-LoRA): an encoder LoRA guided by a fidelity-oriented high-frequency alignment loss to recover authentic details, and a decoder LoRA driven by a perception-oriented loss to synthesize realistic textures. Both LoRA modules are trained via alternating optimization with selective gradient propagation to preserve the pretrained latent structure.At inference, a tunable parameter α enables flexible fidelity-perception trade-offs.Experiments show Latent Harmony achieves state-of-the-art performance across UHD and standard-resolution tasks, effectively balancing efficiency, perceptual quality, and reconstruction accuracy.