T3-S2S: Training-free Triplet Tuning for Sketch to Scene Synthesis in Controllable Concept Art Generation
Not on the map. Why?
Abstract (arXiv)
2D concept art generation for 3D scenes is a crucial yet challenging task in computer graphics, as creating natural intuitive environments still demands extensive manual effort in concept design. While generative AI has simplified 2D concept design via text-to-image synthesis, it struggles with complex multi-instance scenes and offers limited support for structured terrain layout. In this paper, we propose a Training-free Triplet Tuning for Sketch-to-Scene (T3-S2S) generation after reviewing the entire cross-attention mechanism. This scheme revitalizes the ControlNet model for detailed multi-instance generation via three key modules: Prompt Balance ensures keyword representation and minimizes the risk of missing critical instances; Characteristic Priority emphasizes sketch-based features by highlighting TopK indices in feature channels; and Dense Tuning refines contour details within instance-related regions of the attention map. Leveraging the controllability of T3-S2S, we also introduce a feature-sharing strategy with dual prompt sets to generate layer-aware isometric and terrain-view representations for the terrain layout. Experiments show that our sketch-to-scene workflow consistently produces multi-instance 2D scenes with details aligned with input prompts.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 32 of its 36 references that are in the map. First posted on arXiv 18 Dec 2024.
Builds on
32 of 36 references.
References · 32
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2017Squeeze-and-Excitation Networkscited by 4,763
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
20 more
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
- 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
- 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
- 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
- 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
- 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 280
- 2020Query-Key Normalization for Transformerscited by 266
- 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
- 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 242
- 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
- 2022ReCo: Region-Controlled Text-to-Image Generationcited by 177
- 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
- 2023Dense Text-to-Image Generation with Attention Modulationcited by 162
- 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 158
- 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
- 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
- 2024Sketch2Scene: Automatic Generation of Interactive 3D Game Scenes from User's Casual Sketchescited by 21
- 2024Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generationcited by 17
- 2023Detector Guidance for Multi-Object Text-to-Image Generationcited by 11
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.