# CosmicMan: A Text-to-Image Foundation Model for Humans

arXiv: https://arxiv.org/abs/2404.01294 · HTML: https://mlsanity.com/paper/2404.01294/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 67 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 30 of 53)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis](https://mlsanity.com/paper/2306.09341/) (2023)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [EVA-CLIP: Improved Training Techniques for CLIP at Scale](https://mlsanity.com/paper/2303.15389/) (2023)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [nocaps: novel object captioning at scale](https://mlsanity.com/paper/1812.08658/) (2018)
- [TextCaps: a Dataset for Image Captioning with Reading Comprehension](https://mlsanity.com/paper/2003.12462/) (2020)

## Built on by (most cited first, 30 of 30)

- [Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities](https://mlsanity.com/paper/2505.02567/) (2025)
- [InfiniteYou: Flexible Photo Recrafting While Preserving Your Identity](https://mlsanity.com/paper/2503.16418/) (2025)
- [MoLE: Enhancing Human-centric Text-to-image Diffusion via Mixture of Low-rank Experts](https://mlsanity.com/paper/2410.23332/) (2024)
- [AGHI-QA: A Subjective-Aligned Dataset and Metric for AI-Generated Human Images](https://mlsanity.com/paper/2504.21308/) (2025)
- [Human-Centric Foundation Models: Perception, Generation and Agentic Modeling](https://mlsanity.com/paper/2502.08556/) (2025)
- [Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation](https://mlsanity.com/paper/2511.17282/) (2025)
- [Stark: Social Long-Term Multi-Modal Conversation with Persona Commonsense Knowledge](https://mlsanity.com/paper/2407.03958/) (2024)
- [MEAT: Multiview Diffusion Model for Human Generation on Megapixels with Mesh Attention](https://mlsanity.com/paper/2503.08664/) (2025)
- [GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data](https://mlsanity.com/paper/2411.18624/) (2024)
- [Human Body Restoration with One-Step Diffusion Model and A New Benchmark](https://mlsanity.com/paper/2502.01411/) (2025)
- [FaceCraft4D: Animated 3D Facial Avatar Generation from a Single Image](https://mlsanity.com/paper/2504.15179/) (2025)
- [AuthFace: Towards Authentic Blind Face Restoration with Face-oriented Generative Diffusion Prior](https://mlsanity.com/paper/2410.09864/) (2024)
- [FairHuman: Boosting Hand and Face Quality in Human Image Generation with Minimum Potential Delay Fairness in Diffusion Models](https://mlsanity.com/paper/2507.02714/) (2025)
- [GeoRelight: Learning Joint Geometrical Relighting and Reconstruction with Flexible Multi-Modal Diffusion Transformers](https://mlsanity.com/paper/2604.20715/) (2026)
- [Learning Joint ID-Textual Representation for ID-Preserving Image Synthesis](https://mlsanity.com/paper/2504.14202/) (2025)
- [Parameter-Efficient Fine-Tuning with Circulant and Diagonal Vectors](https://mlsanity.com/paper/2505.00580/) (2025)
- [UniMC: Taming Diffusion Transformer for Unified Keypoint-Guided Multi-Class Image Generation](https://mlsanity.com/paper/2507.02713/) (2025)
- [MultiGO++: Monocular 3D Clothed Human Reconstruction via Geometry-Texture Collaboration](https://mlsanity.com/paper/2603.04993/) (2026)
- [Safety Without Semantic Disruptions: Editing-free Safe Image Generation via Context-preserving Dual Latent Reconstruction](https://mlsanity.com/paper/2411.13982/) (2024)
- [RealisID: Scale-Robust and Fine-Controllable Identity Customization via Local and Global Complementation](https://mlsanity.com/paper/2412.16832/) (2024)
- [Unify3D: An Augmented Holistic End-to-end Monocular 3D Human Reconstruction via Anatomy Shaping and Twins Negotiating](https://mlsanity.com/paper/2504.18215/) (2025)
- [ProxT2I: Efficient Reward-Guided Text-to-Image Generation via Proximal Diffusion](https://mlsanity.com/paper/2511.18742/) (2025)
- [SkeleGuide: Explicit Skeleton Reasoning for Context-Aware Human-in-Place Image Synthesis](https://mlsanity.com/paper/2603.01579/) (2026)
- [Towards Anatomically Plausible Human Image Generation via Synthetic Localized Preferences](https://mlsanity.com/paper/2605.25759/) (2026)
- [Human Multi-View Synthesis from a Single-View Model:Transferred Body and Face Representations](https://mlsanity.com/paper/2412.03011/) (2024)
- [Fashionability-Enhancing Outfit Image Editing with Conditional Diffusion Models](https://mlsanity.com/paper/2412.18421/) (2024)
- [From Wardrobe to Canvas: Wardrobe Polyptych LoRA for Part-level Controllable Human Image Generation](https://mlsanity.com/paper/2507.10217/) (2025)
- [From Events to Clarity: The Event-Guided Diffusion Framework for Dehazing](https://mlsanity.com/paper/2511.11944/) (2025)
- [FashionMAC: Deformation-Free Fashion Image Generation with Fine-Grained Model Appearance Customization](https://mlsanity.com/paper/2511.14031/) (2025)
- [CoTZero: Annotation-Free Human-Like Vision Reasoning via Hierarchical Synthetic CoT](https://mlsanity.com/paper/2602.08339/) (2026)

## Abstract (arXiv)

We present CosmicMan, a text-to-image foundation model specialized for generating high-fidelity human images. Unlike current general-purpose foundation models that are stuck in the dilemma of inferior quality and text-image misalignment for humans, CosmicMan enables generating photo-realistic human images with meticulous appearance, reasonable structure, and precise text-image alignment with detailed dense descriptions. At the heart of CosmicMan's success are the new reflections and perspectives on data and models: (1) We found that data quality and a scalable data production flow are essential for the final results from trained models. Hence, we propose a new data production paradigm, Annotate Anyone, which serves as a perpetual data flywheel to produce high-quality data with accurate yet cost-effective annotations over time. Based on this, we constructed a large-scale dataset, CosmicMan-HQ 1.0, with 6 Million high-quality real-world human images in a mean resolution of 1488x1255, and attached with precise text annotations deriving from 115 Million attributes in diverse granularities. (2) We argue that a text-to-image foundation model specialized for humans must be pragmatic -- easy to integrate into down-streaming tasks while effective in producing high-quality human images. Hence, we propose to model the relationship between dense text descriptions and image pixels in a decomposed manner, and present Decomposed-Attention-Refocusing (Daring) training framework. It seamlessly decomposes the cross-attention features in existing text-to-image diffusion model, and enforces attention refocusing without adding extra modules. Through Daring, we show that explicitly discretizing continuous text space into several basic groups that align with human body structure is the key to tackling the misalignment problem in a breeze.