mlsanityData to 14 Oct 2026

Illustrating Classic Brazilian Books using a Text-To-Image Diffusion Model

arXiv 2408.00544 ↗First posted 1 Aug 2024cs.AIcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

In recent years, Generative Artificial Intelligence (GenAI) has undergone a profound transformation in addressing intricate tasks involving diverse modalities such as textual, auditory, visual, and pictorial generation. Within this spectrum, text-to-image (TTI) models have emerged as a formidable approach to generating varied and aesthetically appealing compositions, spanning applications from artistic creation to realistic facial synthesis, and demonstrating significant advancements in computer vision, image processing, and multimodal tasks. The advent of Latent Diffusion Models (LDMs) signifies a paradigm shift in the domain of AI capabilities. This article delves into the feasibility of employing the Stable Diffusion LDM to illustrate literary works. For this exploration, seven classic Brazilian books have been selected as case studies. The objective is to ascertain the practicality of this endeavor and to evaluate the potential of Stable Diffusion in producing illustrations that augment and enrich the reader's experience. We will outline the beneficial aspects, such as the capacity to generate distinctive and contextually pertinent images, as well as the drawbacks, including any shortcomings in faithfully capturing the essence of intricate literary depictions. Through this study, we aim to provide a comprehensive assessment of the viability and efficacy of utilizing AI-generated illustrations in literary contexts, elucidating both the prospects and challenges encountered in this pioneering application of technology.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 13 of its 28 references that are in the map. First posted on arXiv 1 Aug 2024.

Builds on

13 of 28 references.

References · 13

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  6. 2016Improved Techniques for Training GANscited by 2,970
  7. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  8. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  9. 2022Diffusion Models in Vision: A Surveycited by 1,015
  10. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  11. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  12. 2017Perceptual Adversarial Networks for Image-to-Image Transformationcited by 42
1 more
  1. 2023The Carbon Emissions of Writing and Illustrating Are Lower for AI than for Humanscited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model
  2. 2024From Pampas to Pixels: Fine-Tuning Diffusion Models for Ga'ucho Heritage
  3. 2024ORACLE: Leveraging Mutual Information for Consistent Character Generation with LoRAs in Diffusion Models
  4. 2022Language Does More Than Describe: On The Lack Of Figurative Speech in Text-To-Image Models
  5. 2024Generative AI in Vision: A Survey on Models, Metrics and Applications
  6. 2023Text-to-image Diffusion Models in Generative AI: A Survey
  7. 2024Text-to-Image Synthesis: A Decade Survey
  8. 2023RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths