mlsanityData to 7 Oct 2026

T3-S2S: Training-free Triplet Tuning for Sketch to Scene Synthesis in Controllable Concept Art Generation

arXiv 2412.13486 ↗First posted 18 Dec 2024cs.CV cs.CL cs.GRcited by 2

Not on the map. Why?

Abstract (arXiv)

2D concept art generation for 3D scenes is a crucial yet challenging task in computer graphics, as creating natural intuitive environments still demands extensive manual effort in concept design. While generative AI has simplified 2D concept design via text-to-image synthesis, it struggles with complex multi-instance scenes and offers limited support for structured terrain layout. In this paper, we propose a Training-free Triplet Tuning for Sketch-to-Scene (T3-S2S) generation after reviewing the entire cross-attention mechanism. This scheme revitalizes the ControlNet model for detailed multi-instance generation via three key modules: Prompt Balance ensures keyword representation and minimizes the risk of missing critical instances; Characteristic Priority emphasizes sketch-based features by highlighting TopK indices in feature channels; and Dense Tuning refines contour details within instance-related regions of the attention map. Leveraging the controllability of T3-S2S, we also introduce a feature-sharing strategy with dual prompt sets to generate layer-aware isometric and terrain-view representations for the terrain layout. Experiments show that our sketch-to-scene workflow consistently produces multi-instance 2D scenes with details aligned with input prompts.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 32 of its 36 references that are in the map. First posted on arXiv 18 Dec 2024.

Builds on

32 of 36 references.

References · 32

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2017Squeeze-and-Excitation Networkscited by 4,763
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  8. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  9. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  10. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  11. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  12. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
20 more
  1. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  2. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  3. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  4. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  5. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  6. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
  7. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 280
  8. 2020Query-Key Normalization for Transformerscited by 266
  9. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  10. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 242
  11. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  12. 2022ReCo: Region-Controlled Text-to-Image Generationcited by 177
  13. 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
  14. 2023Dense Text-to-Image Generation with Attention Modulationcited by 162
  15. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 158
  16. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
  17. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
  18. 2024Sketch2Scene: Automatic Generation of Interactive 3D Game Scenes from User's Casual Sketchescited by 21
  19. 2024Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generationcited by 17
  20. 2023Detector Guidance for Multi-Object Text-to-Image Generationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.