# The Domain Is a Residue: Adapting Self-Supervised Features, Not Generators

arXiv: https://arxiv.org/abs/2609.37330 · HTML: https://mlsanity.com/paper/2609.37330/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 43 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 30 of 37)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning](https://mlsanity.com/paper/2411.04983/) (2024)
- [Diffusion Transformers with Representation Autoencoders](https://mlsanity.com/paper/2510.11690/) (2025)
- [Splicing ViT Features for Semantic Appearance Transfer](https://mlsanity.com/paper/2201.00424/) (2022)
- [UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.05358/) (2021)
- [One-Step Image Translation with Text-to-Image Models](https://mlsanity.com/paper/2403.12036/) (2024)
- [Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control](https://mlsanity.com/paper/2503.14492/) (2025)
- [Latent Diffusion Model without Variational Autoencoder](https://mlsanity.com/paper/2510.15301/) (2025)
- [SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation](https://mlsanity.com/paper/2503.09641/) (2025)
- [Ensembling Off-the-shelf Models for GAN Training](https://mlsanity.com/paper/2112.09130/) (2021)
- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [Improved Baselines with Representation Autoencoders](https://mlsanity.com/paper/2605.18324/) (2026)
- [Precise Synthetic Image and LiDAR (PreSIL) Dataset for Autonomous Vehicle Perception](https://mlsanity.com/paper/1905.00160/) (2019)
- [One Layer Is Enough: Adapting Pretrained Visual Encoders for Image Generation](https://mlsanity.com/paper/2512.07829/) (2025)
- [Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models](https://mlsanity.com/paper/2605.06388/) (2026)
- [WeatherBench: A Real-World Benchmark Dataset for All-in-One Adverse Weather Image Restoration](https://mlsanity.com/paper/2509.11642/) (2025)
- [Multiplayer Interactive World Models with Representation Autoencoders](https://mlsanity.com/paper/2607.05352/) (2026)
- [Exploring Explicit Domain Supervision for Latent Space Disentanglement in Unpaired Image-to-Image Translation](https://mlsanity.com/paper/1902.03782/) (2019)
- [UVCGAN v2: An Improved Cycle-Consistent GAN for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2303.16280/) (2023)
- [PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion](https://mlsanity.com/paper/2605.23902/) (2026)
- [Driving with DINO: Vision Foundation Features as a Unified Bridge for Sim-to-Real Generation in Autonomous Driving](https://mlsanity.com/paper/2602.06159/) (2026)
- [REGEN: Real-Time Photorealism Enhancement in Games via a Dual-Stage Generative Network Framework](https://mlsanity.com/paper/2508.17061/) (2025)
- [AdapterTune: Zero-Initialized Low-Rank Adapters for Frozen Vision Transformers](https://mlsanity.com/paper/2603.14706/) (2026)

## Abstract (arXiv)

Clearing fog, rain or snow from footage, or turning renders into photographs, must remove the source domain and keep the scene. Unpaired translators carry it through because their generator sees the source appearance (pixels, a near-invertible latent or a control map) and keeps it. A DINO feature map fixes what is in the scene and carries weather, lighting and rendering style as a residue of 13 to 14% of the feature norm. We propose the Representation Feature Adapter (RFA), a 2.9M-parameter network that moves this residue. We train only the adapter and its discriminators; the encoder and a feature-conditioned decoder, trained once for all conditions, stay frozen. Against CycleGAN-Turbo it is ahead on both metrics on fog and on KID on night, and level within noise on snow, rain and haze. On sim-to-real it leads REGEN and HyPER-GAN on both metrics. Only the RFA removes the rain while keeping the scene. The removal costs scene structure: CycleGAN-Turbo keeps more on every condition but fog. On VAE latents the identical adapter collapses to the identity, and decoders from other groups that never saw it render its output. The RFA has about 160 times fewer trainable parameters than CycleGAN-Turbo and under a fifth of its per-condition training time.