mlsanityData to 7 Oct 2026

InstaFormer: Instance-Aware Image-to-Image Translation with Transformer

arXiv 2203.16248 ↗First posted 30 Mar 2022cs.CVcited by 28

Not on the map. Why?

Abstract (arXiv)

We present a novel Transformer-based network architecture for instance-aware image-to-image translation, dubbed InstaFormer, to effectively integrate global- and instance-level information. By considering extracted content features from an image as tokens, our networks discover global consensus of content features by considering context information through a self-attention module in Transformers. By augmenting such tokens with an instance-level feature extracted from the content feature with respect to bounding box information, our framework is capable of learning an interaction between object instances and the global image, thus boosting the instance-awareness. We replace layer normalization (LayerNorm) in standard Transformers with adaptive instance normalization (AdaIN) to enable a multi-modal translation with style codes. In addition, to improve the instance-awareness and translation quality at object regions, we present an instance-level content contrastive loss defined between input and translated image. We conduct experiments to demonstrate the effectiveness of our InstaFormer over the latest methods and provide extensive ablation studies.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 61 of its 71 references that are in the map. First posted on arXiv 30 Mar 2022.

Builds on

61 of 71 references.

References · 61

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2017Mask R-CNNcited by 7,259
  8. 2022Generative Adversarial Networkscited by 5,603
  9. 2016Layer Normalizationcited by 5,579
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  12. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
38 more
  1. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  2. 2016Improved Techniques for Training GANscited by 2,938
  3. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  4. 2014Conditional Generative Adversarial Netscited by 2,152
  5. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  6. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  7. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  8. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  9. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  10. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  11. 2018On the Spectral Bias of Neural Networkscited by 1,321
  12. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  13. 2016Colorful Image Colorizationcited by 1,113
  14. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  15. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
  16. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  17. 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 902
  18. 2020Pre-Trained Image Processing Transformercited by 891
  19. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  20. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  21. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
  22. 2017Unsupervised Image-to-Image Translation Networkscited by 602
  23. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  24. 2021Early Convolutions Help Transformers See Bettercited by 395
  25. 2021Pay Attention to MLPscited by 367
  26. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  27. 2017DualGAN: Unsupervised Dual Learning for Image-to-Image Translationcited by 310
  28. 2017Learning to Discover Cross-Domain Relations with Generative Adversarial Networkscited by 278
  29. 2019DRIT++: Diverse Image-to-Image Translation via Disentangled Representationscited by 244
  30. 2018Pros and Cons of GAN Evaluation Measurescited by 224
  31. 2018Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptationcited by 218
  32. 2017Real-Time User-Guided Image Colorization with Learned Deep Priorscited by 123
  33. 2019Diversify and Match: A Domain Adaptive Representation Learning Paradigm for Object Detectioncited by 122
  34. 2021Generative Adversarial Transformerscited by 120
  35. 2020Cross-domain Correspondence Learning for Exemplar-based Image Translationcited by 116
  36. 2021ViTGAN: Training GANs with Vision Transformerscited by 110
  37. 2020Feature Pyramid Transformercited by 94
  38. 2021The Spatially-Correlative Loss for Various Image Translation Taskscited by 66

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 13; 2024: 6; 2025: 5; 2026: 1.

28 papers.

Sort:

By area:

  1. 2023Milestones in Autonomous Driving and Intelligent Vehicles Part I: Control, Computing System Design, Communication, HD Map, Testing, and Human Behaviorscited by 25
  2. 2025DiffV2IR: Visible-to-Infrared Diffusion Model via Vision-Language Understandingcited by 16
  3. 2023Transformer-based Generative Adversarial Networks in Computer Vision: A Comprehensive Surveycited by 10
  4. 2024C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generationcited by 8
  5. 2022Anatomically constrained CT image translation for heterogeneous blood vessel segmentationcited by 6
  6. 2023One-for-All: Towards Universal Domain Translation with a Single StyleGANcited by 6
  7. 2024Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformerscited by 6
  8. 2024Transformer-based Image and Video Inpainting: Current Challenges and Future Directionscited by 6
  9. 2026Bridging Day and Night: Target-Class Hallucination Suppression in Unpaired Image Translationcited by 6
  10. 2022LANIT: Language-Driven Image-to-Image Translation for Unlabeled Datacited by 5
  11. 2023SHUNIT: Style Harmonization for Unpaired Image-to-Image Translationcited by 3
  12. 2023RADIO: Reference-Agnostic Dubbing Video Synthesiscited by 2
16 more
  1. 2023Masked Discriminators for Content-Consistent Unpaired Image-to-Image Translationcited by 2
  2. 2024Improving 2D-3D Dense Correspondences with Diffusion Models for 6D Object Pose Estimationcited by 2
  3. 2025CSHNet: A Novel Information Asymmetric Image Translation Methodcited by 2
  4. 2022DSI2I: Dense Style for Unpaired Image-to-Image Translationcited by 1
  5. 2023Panoramic Image-to-Image Translationcited by 1
  6. 2024Reliable Multi-modal Medical Image-to-image Translation Independent of Pixel-wise Aligned Datacited by 1
  7. 2024Improving Object Detection via Local-global Contrastive Learningcited by 1
  8. 2025LatticeVision: Image to Image Networks for Modeling Non-Stationary Spatial Datacited by 1
  9. 2023AptSim2Real: Approximately-Paired Sim-to-Real Image Translationcited by 0
  10. 2023Domain Agnostic Image-to-image Translation using Low-Resolution Conditioningcited by 0
  11. 2023Enhancing Visual Domain Adaptation with Source Preparationcited by 0
  12. 2023Learning Nuclei Representations with Masked Image Modellingcited by 0
  13. 2023Disentangled Contrastive Image Translation for Nighttime Surveillancecited by 0
  14. 2023Hierarchical Conditional Semi-Paired Image-to-Image Translation For Multi-Task Image Defect Correction On Shopping Websitescited by 0
  15. 2025DT-NeRF: A Diffusion and Transformer-Based Optimization Approach for Neural Radiance Fields in 3D Reconstructioncited by 0
  16. 2025BrainPuzzle: Hybrid Physics and Data-Driven Reconstruction for Transcranial Ultrasound Tomographycited by 0