# Maybe you don't need a U-Net: convolutional feature upsampling for materials micrograph segmentation

arXiv: https://arxiv.org/abs/2508.21529 · HTML: https://mlsanity.com/paper/2508.21529/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 29 Aug 2025.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [FeatUp: A Model-Agnostic Framework for Features at Any Resolution](https://mlsanity.com/paper/2403.10516/) (2024)
- [Improving 2D Feature Representations by 3D-Aware Fine-Tuning](https://mlsanity.com/paper/2407.20229/) (2024)
- [Denoising Vision Transformers](https://mlsanity.com/paper/2401.02957/) (2024)
- [LiFT: A Surprisingly Simple Lightweight Feature Transform for Dense ViT Descriptors](https://mlsanity.com/paper/2403.14625/) (2024)
- [LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Models](https://mlsanity.com/paper/2504.14032/) (2025)
- [A Generalization of Otsu's Method and Minimum Error Thresholding](https://mlsanity.com/paper/2007.07350/) (2020)
- [Upsampling DINOv2 features for unsupervised vision tasks and weakly supervised materials segmentation](https://mlsanity.com/paper/2410.19836/) (2024)
- [SAMBA: A Trainable Segmentation Web-App with Smart Labelling](https://mlsanity.com/paper/2312.04197/) (2023)
- [Benchmarking Feature Upsampling Methods for Vision Foundation Models using Interactive Segmentation](https://mlsanity.com/paper/2505.02075/) (2025)

## Built on by (most cited first, 2 of 2)

- [What DINO saw: ALiBi positional encoding reduces positional bias in Vision Transformers](https://mlsanity.com/paper/2603.16840/) (2026)
- [Exemplar: Classical Priors Complement Frozen Features for Few-Shot Microscopy Segmentation at Native Resolution](https://mlsanity.com/paper/2609.03080/) (2026)

## Abstract (arXiv)

Feature foundation models - usually vision transformers - offer rich semantic descriptors of images, useful for downstream tasks such as (interactive) segmentation and object detection. For computational efficiency these descriptors are often patch-based, and so struggle to represent the fine features often present in micrographs; they also struggle with the large image sizes present in materials and biological image analysis. In this work, we train a convolutional neural network to upsample low-resolution (i.e, large patch size) foundation model features with reference to the input image. We apply this upsampler network (without any further training) to efficiently featurise and then segment a variety of microscopy images, including plant cells, a lithium-ion battery cathode and organic crystals. The richness of these upsampled features admits separation of hard to segment phases, like hairline cracks. We demonstrate that interactive segmentation with these deep features produces high-quality segmentations far faster and with far fewer labels than training or finetuning a more traditional convolutional network.