# Spatially-Adaptive Pixelwise Networks for Fast Image Translation

arXiv: https://arxiv.org/abs/2012.02992 · HTML: https://mlsanity.com/paper/2012.02992/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2020.

## Built on by (most cited first, 30 of 45)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [Any-resolution Training for High-resolution Image Synthesis](https://mlsanity.com/paper/2204.07156/) (2022)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models](https://mlsanity.com/paper/2211.02048/) (2022)
- [HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworks](https://mlsanity.com/paper/2305.06382/) (2023)
- [Implicit Neural Spatial Representations for Time-dependent PDEs](https://mlsanity.com/paper/2210.00124/) (2022)
- [CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying](https://mlsanity.com/paper/2303.08524/) (2023)
- [High Quality Segmentation for Ultra High-resolution Images](https://mlsanity.com/paper/2111.14482/) (2021)
- [HyperTime: Implicit Neural Representation for Time Series](https://mlsanity.com/paper/2208.05836/) (2022)
- [Dense Pixel-to-Pixel Harmonization via Continuous Image Representation](https://mlsanity.com/paper/2303.01681/) (2023)
- [Neural Image Representations for Multi-Image Fusion and Layer Separation](https://mlsanity.com/paper/2108.01199/) (2021)
- [Implicit Neural Representation Learning for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2112.10541/) (2021)
- [Robustness via Uncertainty-aware Cycle Consistency](https://mlsanity.com/paper/2110.12467/) (2021)
- [Quantum Implicit Neural Representations](https://mlsanity.com/paper/2406.03873/) (2024)
- [Neural Vector Fields for Implicit Surface Representation and Inference](https://mlsanity.com/paper/2204.06552/) (2022)
- [Revisiting Implicit Neural Representations in Low-Level Vision](https://mlsanity.com/paper/2304.10250/) (2023)
- [Generative AI Enables Medical Image Segmentation in Ultra Low-Data Regimes](https://mlsanity.com/paper/2408.17421/) (2024)
- [C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generation](https://mlsanity.com/paper/2411.10788/) (2024)
- [The Perception-Robustness Tradeoff in Deterministic Image Restoration](https://mlsanity.com/paper/2311.09253/) (2023)
- [Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method](https://mlsanity.com/paper/2401.11960/) (2024)
- [GamutMLP: A Lightweight MLP for Color Loss Recovery](https://mlsanity.com/paper/2304.11743/) (2023)
- [Semi-supervised reference-based sketch extraction using a contrastive learning framework](https://mlsanity.com/paper/2407.14026/) (2024)
- [Rethinking Implicit Neural Representations for Vision Learners](https://mlsanity.com/paper/2211.12040/) (2022)
- [CHIMLE: Conditional Hierarchical IMLE for Multimodal Conditional Image Synthesis](https://mlsanity.com/paper/2211.14286/) (2022)
- [Multi-Curve Translator for High-Resolution Photorealistic Image Translation](https://mlsanity.com/paper/2203.07756/) (2022)
- [ASSET: Autoregressive Semantic Scene Editing with Transformers at High Resolutions](https://mlsanity.com/paper/2205.12231/) (2022)
- [DRHDR: A Dual branch Residual Network for Multi-Bracket High Dynamic Range Imaging](https://mlsanity.com/paper/2206.04124/) (2022)
- [K-Space Transformer for Undersampled MRI Reconstruction](https://mlsanity.com/paper/2206.06947/) (2022)
- [CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translation](https://mlsanity.com/paper/2309.03320/) (2023)

## Abstract (arXiv)

We introduce a new generator architecture, aimed at fast and efficient high-resolution image-to-image translation. We design the generator to be an extremely lightweight function of the full-resolution image. In fact, we use pixel-wise networks; that is, each pixel is processed independently of others, through a composition of simple affine transformations and nonlinearities. We take three important steps to equip such a seemingly simple function with adequate expressivity. First, the parameters of the pixel-wise networks are spatially varying so they can represent a broader function class than simple 1x1 convolutions. Second, these parameters are predicted by a fast convolutional network that processes an aggressively low-resolution representation of the input; Third, we augment the input image with a sinusoidal encoding of spatial coordinates, which provides an effective inductive bias for generating realistic novel high-frequency image content. As a result, our model is up to 18x faster than state-of-the-art baselines. We achieve this speedup while generating comparable visual quality across different image resolutions and translation domains.