# Neural inverse procedural modeling of knitting yarns from images

arXiv: https://arxiv.org/abs/2303.00154 · HTML: https://mlsanity.com/paper/2303.00154/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 60 references that are in the map.
- First posted on arXiv 1 Mar 2023.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact](https://mlsanity.com/paper/2106.05306/) (2021)
- [Cloth in the Wind: A Case Study of Physical Measurement through Simulation](https://mlsanity.com/paper/2003.05065/) (2020)
- [Fine-grained differentiable physics: a yarn-level model for fabrics](https://mlsanity.com/paper/2202.00504/) (2022)
- [Neural Inverse Knitting: From Images to Manufacturing Instructions](https://mlsanity.com/paper/1902.02752/) (2019)
- [Smart Fashion: A Review of AI Applications in the Fashion & Apparel Industry](https://mlsanity.com/paper/2111.00905/) (2021)

## Built on by (most cited first, 2 of 2)

- [Single-view 3D reconstruction via inverse procedural modeling](https://mlsanity.com/paper/2310.13373/) (2023)
- [Knitting Robots: A Deep Learning Approach for Reverse-Engineering Fabric Patterns](https://mlsanity.com/paper/2504.14007/) (2025)

## Abstract (arXiv)

We investigate the capabilities of neural inverse procedural modeling to infer high-quality procedural yarn models with fiber-level details from single images of depicted yarn samples. While directly inferring all parameters of the underlying yarn model based on a single neural network may seem an intuitive choice, we show that the complexity of yarn structures in terms of twisting and migration characteristics of the involved fibers can be better encountered in terms of ensembles of networks that focus on individual characteristics. We analyze the effect of different loss functions including a parameter loss to penalize the deviation of inferred parameters to ground truth annotations, a reconstruction loss to enforce similar statistics of the image generated for the estimated parameters in comparison to training images as well as an additional regularization term to explicitly penalize deviations between latent codes of synthetic images and the average latent code of real images in the latent space of the encoder. We demonstrate that the combination of a carefully designed parametric, procedural yarn model with respective network ensembles as well as loss functions even allows robust parameter inference when solely trained on synthetic data. Since our approach relies on the availability of a yarn database with parameter annotations and we are not aware of such a respectively available dataset, we additionally provide, to the best of our knowledge, the first dataset of yarn images with annotations regarding the respective yarn parameters. For this purpose, we use a novel yarn generator that improves the realism of the produced results over previous approaches.