# BATINet: Background-Aware Text to Image Synthesis and Manipulation Network

arXiv: https://arxiv.org/abs/2308.05921 · HTML: https://mlsanity.com/paper/2308.05921/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 18 references that are in the map.
- First posted on arXiv 11 Aug 2023.

## Builds on (most cited first, 15 of 15)

- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis](https://mlsanity.com/paper/1904.01310/) (2019)
- [DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis](https://mlsanity.com/paper/2008.05865/) (2020)
- [ManiGAN: Text-Guided Image Manipulation](https://mlsanity.com/paper/1912.06203/) (2019)
- [Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting](https://mlsanity.com/paper/2005.09704/) (2020)
- [Harmonizer: Learning to Perform White-Box Image and Video Harmonization](https://mlsanity.com/paper/2207.01322/) (2022)
- [Recurrent Affine Transformation for Text-to-image Synthesis](https://mlsanity.com/paper/2204.10482/) (2022)
- [MC-GAN: Multi-conditional Generative Adversarial Network for Image Synthesis](https://mlsanity.com/paper/1805.01123/) (2018)
- [Interactive Image Manipulation with Complex Text Instructions](https://mlsanity.com/paper/2211.15352/) (2022)

## Built on by (most cited first, 2 of 2)

- [TKG-DM: Training-free Chroma Key Content Generation Diffusion Model](https://mlsanity.com/paper/2411.15580/) (2024)
- [Edge-based Denoising Image Compression](https://mlsanity.com/paper/2409.10978/) (2024)

## Abstract (arXiv)

Background-Induced Text2Image (BIT2I) aims to generate foreground content according to the text on the given background image. Most studies focus on generating high-quality foreground content, although they ignore the relationship between the two contents. In this study, we analyzed a novel Background-Aware Text2Image (BAT2I) task in which the generated content matches the input background. We proposed a Background-Aware Text to Image synthesis and manipulation Network (BATINet), which contains two key components: Position Detect Network (PDN) and Harmonize Network (HN). The PDN detects the most plausible position of the text-relevant object in the background image. The HN harmonizes the generated content referring to background style information. Finally, we reconstructed the generation network, which consists of the multi-GAN and attention module to match more user preferences. Moreover, we can apply BATINet to text-guided image manipulation. It solves the most challenging task of manipulating the shape of an object. We demonstrated through qualitative and quantitative evaluations on the CUB dataset that the proposed model outperforms other state-of-the-art methods.