3DrawAgent: Teaching LLM to Draw in 3D with Early Contrastive Experience
Not on the map. Why?
Abstract (arXiv)
Sketching in 3D space enables expressive reasoning about shape, structure, and spatial relationships, yet generating 3D sketches through natural language remains a major challenge. In this work, we introduce 3DrawAgent, a training-free, language-driven framework for 3D sketch generation that leverages large language models (LLMs) to sequentially draw 3D Bezier curves under geometric feedback. Unlike prior 2D sketch agents, our method introduces a relative experience optimization strategy that adapts the recently proposed Group Reward Policy Optimization (GRPO) paradigm. Instead of relying on explicit ground-truth supervision, we construct pairwise comparisons among generated sketches, with each pair consisting of a relatively better and a worse result based on CLIP-based perceptual rewards and LLM-based fine-grained qualitative assessment. These experiences are then used to iteratively refine the prior knowledge of 3D drawing, enabling black-box reinforcement of the model's 3D awareness. This design allows our model to self-improve its spatial understanding and drawing quality without parameter updates. Experiments show that 3DrawAgent can generate complex and coherent 3D Bezier sketches from diverse textual prompts, exhibit emergent geometric reasoning, and generalize to novel shapes, establishing a new paradigm for advancing the field of training-free 3D sketch intelligence.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 39 references that are in the map. First posted on arXiv 9 Apr 2026.
Builds on
29 of 39 references.
References · 29
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
17 more
- 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
- 2017A Neural Representation of Sketch Drawingscited by 311
- 2021DeepCAD: A Deep Generative Network for Computer-Aided Design Modelscited by 266
- 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
- 2020DeepSVG: A Hierarchical Generative Network for Vector Graphics Animationcited by 158
- 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
- 2020SketchGraphs: A Large-Scale Dataset for Modeling Relational Geometry in Computer-Aided Designcited by 72
- 2019Learning to Reconstruct 3D Manhattan Wireframes from a Single Imagecited by 35
- 2021PC2WF: 3D Wireframe Reconstruction from Raw Point Cloudscited by 33
- 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 30
- 20243Doodle: Compact Abstraction of Objects with 3D Strokescited by 21
- 2024EdgeGaussians -- 3D Edge Mapping via Gaussian Splattingcited by 10
- 2024Diff3DS: Generating View-Consistent 3D Sketch via Differentiable Curve Renderingcited by 9
- 2025Curve-Aware Gaussian Splatting for 3D Parametric Curve Reconstructioncited by 9
- 2025SketchSplat: 3D Edge Reconstruction via Differentiable Multi-view Sketch Splattingcited by 5
- 2025ViewCraft3D: High-Fidelity and View-Consistent 3D Vector Graphics Synthesiscited by 5
- 2025Empowering Vector Graphics with Consistently Arbitrary Viewing and View-dependent Visibilitycited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 20254-Doodle: Text to 3D Sketches that Move!
- 2024SketchDream: Sketch-based Text-to-3D Generation and Editing
- 2025LL3M: Large Language 3D Modelers
- 2025PASTA: Part-Aware Sketch-to-3D Shape Generation with Text-Aligned Prior
- 2025End-to-End Fine-Tuning of 3D Texture Generation using Differentiable Rewards
- 2026Socratic-Geo: Synthetic Data Generation and Geometric Reasoning via Multi-Agent Interaction
- 2025Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation
- 2024Sketch2NeRF: Multi-view Sketch-guided Text-to-3D Generation