mlsanityData to 7 Oct 2026

Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought

arXiv 2601.21081 ↗First posted 28 Jan 2026cs.CVcited by 1

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Multimodal models for text-to-image generation have achieved strong visual fidelity, yet they remain brittle under compositional structural constraints, notably generative numeracy, attribute binding, and part-level relations. To address these challenges, we propose Shape-of-Thought (SoT), a visual CoT framework for process-supervised progressive shape assembly in the rendered 2D domain, without external engines at inference time. SoT trains a unified multimodal autoregressive model to generate interleaved textual plans and rendered intermediate states, helping the model capture shape-assembly logic without producing explicit geometric representations. Unlike text-only CoT, each decision is grounded in a rendered state, making counts, attachments, topology, and intermediate part-addition errors inspectable across the trajectory. To support this paradigm, we introduce SoT-26K, a large-scale dataset of grounded assembly traces derived from part-based CAD hierarchies, and T2S-CompBench, a benchmark for evaluating structural integrity and trace faithfulness. Fine-tuning on SoT-26K achieves 88.4% on component numeracy and 84.8% on structural topology, outperforming direct generation by +24.2 points on component numeracy and +19.3 points on structural topology. SoT establishes a transparent testbed for rendered-domain structure-aware generation. The code is available at https://github.com/yuhuo03/Shape-of-Thought.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 22 of its 47 references that are in the map. First posted on arXiv 28 Jan 2026.

Builds on

22 of 47 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning30 Oct
  2. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey16 Mar

Other references · 20

  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  2. 2024GPT-4o System Cardcited by 4,467
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  6. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  7. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  8. 2025SAM 3: Segment Anything with Conceptscited by 835
  9. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  10. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  11. 2023Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuningcited by 168
  12. 2025Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoningcited by 67
8 more
  1. 2023T3Bench: Benchmarking Current Progress in Text-to-3D Generationcited by 56
  2. 2025FlowVLA: Visual Chain of Thought-based Motion Reasoning for Vision-Language-Action Modelscited by 50
  3. 20243D-PreMise: Can Large Language Models Generate 3D Shapes with Sharp Features and Parametric Control?cited by 21
  4. 2025OpenHOI: Open-World Hand-Object Interaction Synthesis with Multimodal Large Language Modelcited by 17
  5. 2025MVGBench: Comprehensive Benchmark for Multi-view Generation Modelscited by 9
  6. 2025SceneCOT: Eliciting Grounded Chain-of-Thought Reasoning in 3D Scenescited by 7
  7. 2025CoT-Pose: Chain-of-Thought Reasoning for 3D Pose Generation from Abstract Promptscited by 2
  8. 2024The Moral Foundations Weibo Corpuscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning
  2. 2025Factuality Matters: When Image Generation and Editing Meet Structured Visuals
  3. 2025Thinking with Generated Images
  4. 2026IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation
  5. 2026CoVA-SFT: A Large-Scale Dataset for Chain of Visual Abstractions
  6. 2023ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Model
  7. 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark
  8. 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing