Illustrious: an Open Advanced Illustration Model
Not on the map. Why?
Abstract (arXiv)
In this work, we share the insights for achieving state-of-the-art quality in our text-to-image anime image generative model, called Illustrious. To achieve high resolution, dynamic color range images, and high restoration ability, we focus on three critical approaches for model improvement. First, we delve into the significance of the batch size and dropout control, which enables faster learning of controllable token based concept activations. Second, we increase the training resolution of images, affecting the accurate depiction of character anatomy in much higher resolution, extending its generation capability over 20MP with proper methods. Finally, we propose the refined multi-level captions, covering all tags and various natural language captions as a critical factor for model development. Through extensive analysis and experiments, Illustrious demonstrates state-of-the-art performance in terms of animation style, outperforming widely-used models in illustration domains, propelling easier customization and personalization with nature of open source. We plan to publicly release updated Illustrious model series sequentially as well as sustainable plans for improvements.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 49 of its 69 references that are in the map. First posted on arXiv 30 Sep 2024.
Builds on
49 of 69 references.
References · 49
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
37 more
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
- 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
- 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
- 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
- 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
- 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
- 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
- 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
- 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
- 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 326
- 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
- 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 285
- 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 246
- 2018Measuring the Effects of Data Parallelism on Neural Network Trainingcited by 190
- 2023Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Modelscited by 148
- 2019Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Modelcited by 116
- 2024Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Modelscited by 113
- 2023Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluationcited by 108
- 2023Input Perturbation Reduces Exposure Bias in Diffusion Modelscited by 88
- 2023Diversity is Definitely Needed: Improving Model-Agnostic Zero-shot Classification via Stable Diffusioncited by 81
- 2017AdaBatch: Adaptive Batch Sizes for Training Deep Neural Networkscited by 29
- 2021DAF:re: A Challenging, Crowd-Sourced, Large-Scale, Long-Tailed Dataset For Anime Character Recognitioncited by 9
- 2024Improving Text Generation on Images with Synthetic Captionscited by 3
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 4.
5 papers.
- 2024TIPO: Text to Image with Text Presampling for Prompt Optimizationcited by 6
- 2026The Quadratic Geometry of Flow Matching: Semantic Granularity Alignment for Text-to-Image Synthesiscited by 0
- 2026Let Triggers Control: Frequency-Aware Dropout for Effective Token Controlcited by 0
- 2026ExpressEdit: Fast Editing of Stylized Facial Expressions with Diffusion Models in Photoshopcited by 0
- 2026FreeStyle: Free Control of Style-Content Dual-Reference Generation from Community LoRA Miningcited by 0
- 2026FreeStyle: Free Control of Style-Content Dual-Reference Generation from Community LoRA Mining18 Jun 2026
- 2026ExpressEdit: Fast Editing of Stylized Facial Expressions with Diffusion Models in Photoshop3 Apr 2026
- 2026Let Triggers Control: Frequency-Aware Dropout for Effective Token Control28 Mar 2026
- 2026The Quadratic Geometry of Flow Matching: Semantic Granularity Alignment for Text-to-Image Synthesis11 Mar 2026
- 2024TIPO: Text to Image with Text Presampling for Prompt Optimization12 Nov 2024