# PolyGen: An Autoregressive Generative Model of 3D Meshes

arXiv: https://arxiv.org/abs/2002.10880 · HTML: https://mlsanity.com/paper/2002.10880/

## Key facts

- Cited by 269 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2020.

## Built on by (most cited first, 30 of 269)

- [Structured 3D Latents for Scalable and Versatile 3D Generation](https://mlsanity.com/paper/2412.01506/) (2024)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets](https://mlsanity.com/paper/2406.13897/) (2024)
- [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022)
- [OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation](https://mlsanity.com/paper/2301.07525/) (2023)
- [One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion](https://mlsanity.com/paper/2311.07885/) (2023)
- [DeepCAD: A Deep Generative Network for Computer-Aided Design Models](https://mlsanity.com/paper/2105.09492/) (2021)
- [MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers](https://mlsanity.com/paper/2311.15475/) (2023)
- [CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation](https://mlsanity.com/paper/2110.02624/) (2021)
- [VectorMapNet: End-to-end Vectorized HD Map Learning](https://mlsanity.com/paper/2206.08920/) (2022)
- [Text2Tex: Text-driven Texture Synthesis via Diffusion Models](https://mlsanity.com/paper/2303.11396/) (2023)
- [ATISS: Autoregressive Transformers for Indoor Scene Synthesis](https://mlsanity.com/paper/2110.03675/) (2021)
- [MeshDiffusion: Score-based Generative 3D Mesh Modeling](https://mlsanity.com/paper/2303.08133/) (2023)
- [XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies](https://mlsanity.com/paper/2312.03806/) (2023)
- [Native and Compact Structured Latents for 3D Generation](https://mlsanity.com/paper/2512.14692/) (2025)
- [MotionLM: Multi-Agent Motion Forecasting as Language Modeling](https://mlsanity.com/paper/2309.16534/) (2023)
- [Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.14704/) (2022)
- [MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers](https://mlsanity.com/paper/2406.10163/) (2024)
- [Flexible Isosurface Extraction for Gradient-Based Mesh Optimization](https://mlsanity.com/paper/2308.05371/) (2023)
- [D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Video](https://mlsanity.com/paper/2205.15838/) (2022)
- [Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation](https://mlsanity.com/paper/2303.07937/) (2023)
- [Recent Advances in 3D Gaussian Splatting](https://mlsanity.com/paper/2403.11134/) (2024)
- [Locally Attentional SDF Diffusion for Controllable 3D Shape Generation](https://mlsanity.com/paper/2305.04461/) (2023)
- [Diffusion-SDF: Text-to-Shape via Voxelized Diffusion](https://mlsanity.com/paper/2212.03293/) (2022)
- [SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks](https://mlsanity.com/paper/2207.04632/) (2022)
- [CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refiner](https://mlsanity.com/paper/2405.14979/) (2024)
- [ShapeFormer: Transformer-based Shape Completion via Sparse Representation](https://mlsanity.com/paper/2201.10326/) (2022)

## Abstract (arXiv)

Polygon meshes are an efficient representation of 3D geometry, and are of central importance in computer graphics, robotics and games development. Existing learning-based approaches have avoided the challenges of working with 3D meshes, instead using alternative object representations that are more compatible with neural architectures and training approaches. We present an approach which models the mesh directly, predicting mesh vertices and faces sequentially using a Transformer-based architecture. Our model can condition on a range of inputs, including object classes, voxels, and images, and because the model is probabilistic it can produce samples that capture uncertainty in ambiguous scenarios. We show that the model is capable of producing high-quality, usable meshes, and establish log-likelihood benchmarks for the mesh-modelling task. We also evaluate the conditional models on surface reconstruction metrics against alternative methods, and demonstrate competitive performance despite not training directly on this task.