# Less-to-More Generalization: Unlocking More Controllability by In-Context Generation

arXiv: https://arxiv.org/abs/2504.02160 · HTML: https://mlsanity.com/paper/2504.02160/

## Key facts

- Cited by 154 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 52 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Builds on (most cited first, 30 of 46)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [Improving alignment of dialogue agents via targeted human judgements](https://mlsanity.com/paper/2209.14375/) (2022)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing](https://mlsanity.com/paper/2305.14720/) (2023)
- [InstantID: Zero-shot Identity-Preserving Generation in Seconds](https://mlsanity.com/paper/2401.07519/) (2024)
- [ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation](https://mlsanity.com/paper/2302.13848/) (2023)
- [OmniGen: Unified Image Generation](https://mlsanity.com/paper/2409.11340/) (2024)
- [Self-critiquing models for assisting human evaluators](https://mlsanity.com/paper/2206.05802/) (2022)

## Built on by (most cited first, 30 of 154)

- [OmniGen2: Towards Instruction-Aligned Multimodal Generation](https://mlsanity.com/paper/2506.18871/) (2025)
- [In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer](https://mlsanity.com/paper/2504.20690/) (2025)
- [Emu3.5: Native Multimodal Models are World Learners](https://mlsanity.com/paper/2510.26583/) (2025)
- [Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generation](https://mlsanity.com/paper/2508.09987/) (2025)
- [Ovis-U1 Technical Report](https://mlsanity.com/paper/2506.23044/) (2025)
- [XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation](https://mlsanity.com/paper/2506.21416/) (2025)
- [VisualCloze: A Universal Image Generation Framework via Visual In-Context Learning](https://mlsanity.com/paper/2504.07960/) (2025)
- [DreamOmni2: Multimodal Instruction-based Editing and Generation](https://mlsanity.com/paper/2510.06679/) (2025)
- [CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design](https://mlsanity.com/paper/2505.19114/) (2025)
- [USO: Unified Style and Subject-Driven Generation via Disentangled and Reward Learning](https://mlsanity.com/paper/2508.18966/) (2025)
- [InstantCharacter: Personalize Any Characters with a Scalable Diffusion Transformer Framework](https://mlsanity.com/paper/2504.12395/) (2025)
- [UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Reward](https://mlsanity.com/paper/2509.06818/) (2025)
- [FullDiT2: Efficient In-Context Conditioning for Video Diffusion Transformers](https://mlsanity.com/paper/2506.04213/) (2025)
- [ContextGen: Contextual Layout Anchoring for Identity-Consistent Multi-Instance Generation](https://mlsanity.com/paper/2510.11000/) (2025)
- [MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models](https://mlsanity.com/paper/2505.19415/) (2025)
- [VINCIE: Unlocking In-context Image Editing from Video](https://mlsanity.com/paper/2506.10941/) (2025)
- [WithAnyone: Towards Controllable and ID Consistent Image Generation](https://mlsanity.com/paper/2510.14975/) (2025)
- [WordCon: Word-level Typography Control in Scene Text Rendering](https://mlsanity.com/paper/2506.21276/) (2025)
- [Cut2Next: Generating Next Shot via In-Context Tuning](https://mlsanity.com/paper/2508.08244/) (2025)
- [Query-Kontext: An Unified Multimodal Model for Image Generation and Editing](https://mlsanity.com/paper/2509.26641/) (2025)
- [MOSAIC: Multi-Subject Personalized Generation via Correspondence-Aware Alignment and Disentanglement](https://mlsanity.com/paper/2509.01977/) (2025)
- [NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generation](https://mlsanity.com/paper/2601.02204/) (2026)
- [Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adapter](https://mlsanity.com/paper/2505.18612/) (2025)
- [LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformers](https://mlsanity.com/paper/2505.23758/) (2025)
- [Lance: Unified Multimodal Modeling by Multi-Task Synergy](https://mlsanity.com/paper/2605.18678/) (2026)
- [Image Editing As Programs with Diffusion Models](https://mlsanity.com/paper/2506.04158/) (2025)
- [IC-Custom: Diverse Image Customization via In-Context Learning](https://mlsanity.com/paper/2507.01926/) (2025)
- [MICo-150K: A Comprehensive Dataset Advancing Multi-Image Composition](https://mlsanity.com/paper/2512.07348/) (2025)
- [Scone: Bridging Composition and Distinction in Subject-Driven Image Generation via Unified Understanding-Generation Modeling](https://mlsanity.com/paper/2512.12675/) (2025)
- [Visual-Aware CoT: Achieving High-Fidelity Visual Consistency in Unified Models](https://mlsanity.com/paper/2512.19686/) (2025)

## Abstract (arXiv)

Although subject-driven generation has been extensively explored in image generation due to its wide applications, it still has challenges in data scalability and subject expansibility. For the first challenge, moving from curating single-subject datasets to multiple-subject ones and scaling them is particularly difficult. For the second, most recent methods center on single-subject generation, making it hard to apply when dealing with multi-subject scenarios. In this study, we propose a highly-consistent data synthesis pipeline to tackle this challenge. This pipeline harnesses the intrinsic in-context generation capabilities of diffusion transformers and generates high-consistency multi-subject paired data. Additionally, we introduce UNO, which consists of progressive cross-modal alignment and universal rotary position embedding. It is a multi-image conditioned subject-to-image model iteratively trained from a text-to-image model. Extensive experiments show that our method can achieve high consistency while ensuring controllability in both single-subject and multi-subject driven generation.