# Bi-level Feature Alignment for Versatile Image Translation and Manipulation

arXiv: https://arxiv.org/abs/2107.03021 · HTML: https://mlsanity.com/paper/2107.03021/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 77 of its 98 references that are in the map.
- First posted on arXiv 7 Jul 2021.

## Builds on (most cited first, 30 of 77)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Learning to Adapt Structured Output Space for Semantic Segmentation](https://mlsanity.com/paper/1802.10349/) (2018)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Universal Style Transfer via Feature Transforms](https://mlsanity.com/paper/1705.08086/) (2017)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [Learning from Simulated and Unsupervised Images through Adversarial Training](https://mlsanity.com/paper/1612.07828/) (2016)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Image to Image Translation for Domain Adaptation](https://mlsanity.com/paper/1712.00479/) (2017)
- [GANimation: Anatomically-aware Facial Animation from a Single Image](https://mlsanity.com/paper/1807.09251/) (2018)
- [Controllable Person Image Synthesis with Attribute-Decomposed GAN](https://mlsanity.com/paper/2003.12267/) (2020)

## Built on by (most cited first, 20 of 20)

- [Semantic Image Synthesis via Diffusion Models](https://mlsanity.com/paper/2207.00050/) (2022)
- [A Comprehensive Survey on Source-free Domain Adaptation](https://mlsanity.com/paper/2302.11803/) (2023)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [WaveFill: A Wavelet-based Generation Network for Image Inpainting](https://mlsanity.com/paper/2107.11027/) (2021)
- [DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation](https://mlsanity.com/paper/2207.06124/) (2022)
- [MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image Translation](https://mlsanity.com/paper/2209.11047/) (2022)
- [Exploring the Naturalness of AI-Generated Images](https://mlsanity.com/paper/2312.05476/) (2023)
- [Towards Counterfactual Image Manipulation via CLIP](https://mlsanity.com/paper/2207.02812/) (2022)
- [VMRF: View Matching Neural Radiance Fields](https://mlsanity.com/paper/2207.02621/) (2022)
- [All-to-key Attention for Arbitrary Style Transfer](https://mlsanity.com/paper/2212.04105/) (2022)
- [BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization](https://mlsanity.com/paper/2212.02268/) (2022)
- [Learning by Sorting: Self-supervised Learning with Group Ordering Constraints](https://mlsanity.com/paper/2301.02009/) (2023)
- [Face Transformer: Towards High Fidelity and Accurate Face Swapping](https://mlsanity.com/paper/2304.02530/) (2023)
- [Auto-regressive Image Synthesis with Integrated Quantization](https://mlsanity.com/paper/2207.10776/) (2022)
- [Hadamard product in deep learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2504.13112/) (2025)
- [EBDM: Exemplar-guided Image Translation with Brownian-bridge Diffusion Models](https://mlsanity.com/paper/2410.09802/) (2024)
- [DragGaussian: Enabling Drag-style Manipulation on 3D Gaussian Representation](https://mlsanity.com/paper/2405.05800/) (2024)
- [DSI2I: Dense Style for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2212.13253/) (2022)
- [Latent Multi-Relation Reasoning for GAN-Prior based Image Super-Resolution](https://mlsanity.com/paper/2208.02861/) (2022)
- [Clutter Detection and Removal by Multi-Objective Analysis for Photographic Guidance](https://mlsanity.com/paper/2507.14553/) (2025)

## Abstract (arXiv)

Generative adversarial networks (GANs) have achieved great success in image translation and manipulation. However, high-fidelity image generation with faithful style control remains a grand challenge in computer vision. This paper presents a versatile image translation and manipulation framework that achieves accurate semantic and style guidance in image generation by explicitly building a correspondence. To handle the quadratic complexity incurred by building the dense correspondences, we introduce a bi-level feature alignment strategy that adopts a top-k operation to rank block-wise features followed by dense attention between block features which reduces memory cost substantially. As the top-k operation involves index swapping which precludes the gradient propagation, we approximate the non-differentiable top-k operation with a regularized earth mover's problem so that its gradient can be effectively back-propagated. In addition, we design a novel semantic position encoding mechanism that builds up coordinate for each individual semantic region to preserve texture structures while building correspondences. Further, we design a novel confidence feature injection module which mitigates mismatch problem by fusing features adaptively according to the reliability of built correspondences. Extensive experiments show that our method achieves superior performance qualitatively and quantitatively as compared with the state-of-the-art.