# Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought

arXiv: https://arxiv.org/abs/2601.21081 · HTML: https://mlsanity.com/paper/2601.21081/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 28 Jan 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 22 of 22)

- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Shap-E: Generating Conditional 3D Implicit Functions](https://mlsanity.com/paper/2305.02463/) (2023)
- [SAM 3D: 3Dfy Anything in Images](https://mlsanity.com/paper/2511.16624/) (2025)
- [Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey](https://mlsanity.com/paper/2503.12605/) (2025)
- [Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning](https://mlsanity.com/paper/2309.02591/) (2023)
- [Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning](https://mlsanity.com/paper/2506.13654/) (2025)
- [T3Bench: Benchmarking Current Progress in Text-to-3D Generation](https://mlsanity.com/paper/2310.02977/) (2023)
- [ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning](https://mlsanity.com/paper/2510.27492/) (2025)
- [FlowVLA: Visual Chain of Thought-based Motion Reasoning for Vision-Language-Action Models](https://mlsanity.com/paper/2508.18269/) (2025)
- [3D-PreMise: Can Large Language Models Generate 3D Shapes with Sharp Features and Parametric Control?](https://mlsanity.com/paper/2401.06437/) (2024)
- [OpenHOI: Open-World Hand-Object Interaction Synthesis with Multimodal Large Language Model](https://mlsanity.com/paper/2505.18947/) (2025)
- [MVGBench: Comprehensive Benchmark for Multi-view Generation Models](https://mlsanity.com/paper/2507.00006/) (2025)
- [SceneCOT: Eliciting Grounded Chain-of-Thought Reasoning in 3D Scenes](https://mlsanity.com/paper/2510.16714/) (2025)
- [CoT-Pose: Chain-of-Thought Reasoning for 3D Pose Generation from Abstract Prompts](https://mlsanity.com/paper/2508.07540/) (2025)
- [The Moral Foundations Weibo Corpus](https://mlsanity.com/paper/2411.09612/) (2024)

## Built on by (most cited first, 1 of 1)

- [Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioning](https://mlsanity.com/paper/2609.00656/) (2026)

## Abstract (arXiv)

Multimodal models for text-to-image generation have achieved strong visual fidelity, yet they remain brittle under compositional structural constraints, notably generative numeracy, attribute binding, and part-level relations. To address these challenges, we propose Shape-of-Thought (SoT), a visual CoT framework for process-supervised progressive shape assembly in the rendered 2D domain, without external engines at inference time. SoT trains a unified multimodal autoregressive model to generate interleaved textual plans and rendered intermediate states, helping the model capture shape-assembly logic without producing explicit geometric representations. Unlike text-only CoT, each decision is grounded in a rendered state, making counts, attachments, topology, and intermediate part-addition errors inspectable across the trajectory. To support this paradigm, we introduce SoT-26K, a large-scale dataset of grounded assembly traces derived from part-based CAD hierarchies, and T2S-CompBench, a benchmark for evaluating structural integrity and trace faithfulness. Fine-tuning on SoT-26K achieves 88.4% on component numeracy and 84.8% on structural topology, outperforming direct generation by +24.2 points on component numeracy and +19.3 points on structural topology. SoT establishes a transparent testbed for rendered-domain structure-aware generation. The code is available at https://github.com/yuhuo03/Shape-of-Thought.