mlsanityData to 14 Oct 2026

Toward Multimodal Image-to-Image Translation

arXiv 1711.11586 ↗First posted 30 Nov 2017cs.CV cs.GR stat.MLcited by 296

Not on the map. Why?

Cited by 30 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Many image-to-image translation problems are ambiguous, as a single input image may correspond to multiple possible outputs. In this work, we aim to model a distribution of possible outputs in a conditional generative modeling setting. The ambiguity of the mapping is distilled in a low-dimensional latent vector, which can be randomly sampled at test time. A generator learns to map the given input, combined with this latent code, to the output. We explicitly encourage the connection between output and the latent code to be invertible. This helps prevent a many-to-one mapping from the latent code to the output during training, also known as the problem of mode collapse, and produces more diverse results. We explore several variants of this approach by employing different training objectives, network architectures, and methods of injecting the latent code. Our proposed method encourages bijective consistency between the latent encoding and output modes. We present a systematic comparison of our method and other variants on both perceptual realism and diversity.

Key facts. Cited by 296 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 120; 2022: 58; 2023: 64; 2024: 27; 2025: 13; 2026: 13.

296 papers.

Sort:

By area:

  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  2. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
  3. 2021GAN Inversion: A Surveycited by 367
  4. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 220
  5. 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuningcited by 208
  6. 2021Few-shot Image Generation via Cross-domain Correspondencecited by 207
  7. 2021GAN Prior Embedded Network for Blind Face Restoration in the Wildcited by 197
  8. 2021A Survey on Neural Speech Synthesiscited by 184
  9. 2022Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pocketscited by 183
  10. 2022Sketch-Guided Text-to-Image Diffusion Modelscited by 180
  11. 2019Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Modelscited by 178
  12. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
38 more
  1. 2022Pretraining is All You Need for Image-to-Image Translationcited by 160
  2. 2021High-Fidelity Pluralistic Image Completion with Transformerscited by 134
  3. 2021UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Modelscited by 133
  4. 2022Semantic Image Synthesis via Diffusion Modelscited by 132
  5. 2022Neural Optimal Transportcited by 126
  6. 2021Image-to-Image Translation: Methods and Applicationscited by 116
  7. 2024One-Step Image Translation with Text-to-Image Modelscited by 116
  8. 2021Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoderscited by 109
  9. 2021GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worldscited by 98
  10. 2021Only a Matter of Style: Age Transformation Using a Style-Based Regression Modelcited by 92
  11. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  12. 2021PD-GAN: Probabilistic Diverse GAN for Image Inpaintingcited by 80
  13. 2023A-STAR: Test-time Attention Segregation and Retention for Text-to-image Synthesiscited by 78
  14. 2024Distilling Diffusion Models into Conditional GANscited by 78
  15. 2021Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentationcited by 76
  16. 2021Multimodal Conditional Image Synthesis with Product-of-Experts GANscited by 75
  17. 2021The Spatially-Correlative Loss for Various Image Translation Taskscited by 66
  18. 2021Image-to-image Translation via Hierarchical Style Disentanglementcited by 63
  19. 2021High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Networkcited by 61
  20. 2021MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Predictioncited by 61
  21. 2021Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Imagescited by 58
  22. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusioncited by 57
  23. 2021Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigationcited by 53
  24. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  25. 2021Diverse Semantic Image Synthesis via Probability Distribution Modelingcited by 49
  26. 2021Stochastic Image-to-Video Synthesis using cINNscited by 49
  27. 2021StyleAlign: Analysis and Applications of Aligned StyleGAN Modelscited by 48
  28. 2021HumanGAN: A Generative Model of Humans Imagescited by 47
  29. 2021Learning Disentangled Representations in the Imaging Domaincited by 46
  30. 2021Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translationcited by 45
  31. 2021Curious Representation Learning for Embodied Intelligencecited by 44
  32. 2023M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimizationcited by 43
  33. 2021iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesiscited by 42
  34. 2021Unbalanced Feature Transport for Exemplar-based Image Translationcited by 41
  35. 2022Diff-Font: Diffusion Model for Robust One-Shot Font Generationcited by 40
  36. 2023GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videoscited by 40
  37. 2021DRB-GAN: A Dynamic ResBlock Generative Adversarial Network for Artistic Style Transfercited by 39
  38. 2022QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translationcited by 39

50 most cited of 296 All 296 (JSON)