# FeatUp: A Model-Agnostic Framework for Features at Any Resolution

arXiv: https://arxiv.org/abs/2403.10516 · HTML: https://mlsanity.com/paper/2403.10516/

## Key facts

- Cited by 116 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 89 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Builds on (most cited first, 30 of 57)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Learning Deconvolution Network for Semantic Segmentation](https://mlsanity.com/paper/1505.04366/) (2015)
- [UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation](https://mlsanity.com/paper/2004.08790/) (2020)

## Built on by (most cited first, 30 of 116)

- [StableNormal: Reducing Diffusion Variance for Stable and Sharp Normal](https://mlsanity.com/paper/2406.16864/) (2024)
- [SAMPart3D: Segment Any Part in 3D Objects](https://mlsanity.com/paper/2411.07184/) (2024)
- [MEt3R: Measuring Multi-View Consistency in Generated Images](https://mlsanity.com/paper/2501.06336/) (2025)
- [Improving 2D Feature Representations by 3D-Aware Fine-Tuning](https://mlsanity.com/paper/2407.20229/) (2024)
- [AnomalyDINO: Boosting Patch-based Few-shot Anomaly Detection with DINOv2](https://mlsanity.com/paper/2405.14529/) (2024)
- [GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understanding](https://mlsanity.com/paper/2412.13193/) (2024)
- [Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generation](https://mlsanity.com/paper/2412.06016/) (2024)
- [MedCLIP-SAMv2: Towards Universal Text-Driven Medical Image Segmentation](https://mlsanity.com/paper/2409.19483/) (2024)
- [AnyUp: Universal Feature Upsampling](https://mlsanity.com/paper/2510.12764/) (2025)
- [LiFT: A Surprisingly Simple Lightweight Feature Transform for Dense ViT Descriptors](https://mlsanity.com/paper/2403.14625/) (2024)
- [Feat2GS: Probing Visual Foundation Models with Gaussian Splatting](https://mlsanity.com/paper/2412.09606/) (2024)
- [SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2410.01768/) (2024)
- [Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentation](https://mlsanity.com/paper/2502.00173/) (2025)
- [LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Models](https://mlsanity.com/paper/2504.14032/) (2025)
- [JAFAR: Jack up Any Feature at Any Resolution](https://mlsanity.com/paper/2506.11136/) (2025)
- [PixOOD: Pixel-Level Out-of-Distribution Detection](https://mlsanity.com/paper/2405.19882/) (2024)
- [Keypoint Abstraction using Large Models for Object-Relative Imitation Learning](https://mlsanity.com/paper/2410.23254/) (2024)
- [Real-Time 4K Super-Resolution of Compressed AVIF Images. AIS 2024 Challenge Survey](https://mlsanity.com/paper/2404.16484/) (2024)
- [Eval3D: Interpretable and Fine-grained Evaluation for 3D Generation](https://mlsanity.com/paper/2504.18509/) (2025)
- [SteerX: Creating Any Camera-Free 3D and 4D Scenes with Geometric Steering](https://mlsanity.com/paper/2503.12024/) (2025)
- [SAM-LAD: Segment Anything Model Meets Zero-Shot Logic Anomaly Detection](https://mlsanity.com/paper/2406.00625/) (2024)
- [SkySplat: Generalizable 3D Gaussian Splatting from Multi-Temporal Sparse Satellite Images](https://mlsanity.com/paper/2508.09479/) (2025)
- [T-3DGS: Removing Transient Objects for 3D Scene Reconstruction](https://mlsanity.com/paper/2412.00155/) (2024)
- [Feed-Forward SceneDINO for Unsupervised Semantic Scene Completion](https://mlsanity.com/paper/2507.06230/) (2025)
- [Zero-Shot Video Semantic Segmentation based on Pre-Trained Diffusion Models](https://mlsanity.com/paper/2405.16947/) (2024)
- [Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledge](https://mlsanity.com/paper/2410.13016/) (2024)
- [INSID3: Training-Free In-Context Segmentation with DINOv3](https://mlsanity.com/paper/2603.28480/) (2026)
- [TIGER: Text-Instructed 3D Gaussian Retrieval and Coherent Editing](https://mlsanity.com/paper/2405.14455/) (2024)
- [DV-Matcher: Deformation-based Non-Rigid Point Cloud Matching Guided by Pre-trained Visual Features](https://mlsanity.com/paper/2408.08568/) (2024)
- [Online Language Splatting](https://mlsanity.com/paper/2503.09447/) (2025)

## Abstract (arXiv)

Deep features are a cornerstone of computer vision research, capturing image semantics and enabling the community to solve downstream tasks even in the zero- or few-shot regime. However, these features often lack the spatial resolution to directly perform dense prediction tasks like segmentation and depth prediction because models aggressively pool information over large areas. In this work, we introduce FeatUp, a task- and model-agnostic framework to restore lost spatial information in deep features. We introduce two variants of FeatUp: one that guides features with high-resolution signal in a single forward pass, and one that fits an implicit model to a single image to reconstruct features at any resolution. Both approaches use a multi-view consistency loss with deep analogies to NeRFs. Our features retain their original semantics and can be swapped into existing applications to yield resolution and performance gains even without re-training. We show that FeatUp significantly outperforms other feature upsampling and image super-resolution approaches in class activation map generation, transfer learning for segmentation and depth prediction, and end-to-end training for semantic segmentation.