mlsanityData to 7 Oct 2026

FeatUp: A Model-Agnostic Framework for Features at Any Resolution

arXiv 2403.10516 ↗First posted 15 Mar 2024cs.CV cs.AI cs.IRcited by 114

Not on the map. Why?

Abstract (arXiv)

Deep features are a cornerstone of computer vision research, capturing image semantics and enabling the community to solve downstream tasks even in the zero- or few-shot regime. However, these features often lack the spatial resolution to directly perform dense prediction tasks like segmentation and depth prediction because models aggressively pool information over large areas. In this work, we introduce FeatUp, a task- and model-agnostic framework to restore lost spatial information in deep features. We introduce two variants of FeatUp: one that guides features with high-resolution signal in a single forward pass, and one that fits an implicit model to a single image to reconstruct features at any resolution. Both approaches use a multi-view consistency loss with deep analogies to NeRFs. Our features retain their original semantics and can be swapped into existing applications to yield resolution and performance gains even without re-training. We show that FeatUp significantly outperforms other feature upsampling and image super-resolution approaches in class activation map generation, transfer learning for segmentation and depth prediction, and end-to-end training for semantic segmentation.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). Builds on 57 of its 89 references that are in the map. First posted on arXiv 15 Mar 2024.

Builds on

57 of 89 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2016Layer Normalizationcited by 5,579
  9. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  10. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  11. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  12. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
38 more
  1. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2017Non-local Neural Networkscited by 2,290
  5. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  6. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  7. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  8. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  9. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  10. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  11. 2017Deep Image Priorcited by 1,097
  12. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  13. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  14. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 550
  15. 2017Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolutioncited by 452
  16. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  17. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  18. 2020UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentationcited by 331
  19. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  20. 2020Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentationcited by 299
  21. 2017"Zero-Shot" Super-Resolution using Deep Internal Learningcited by 285
  22. 2019Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relationscited by 245
  23. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  24. 2017Deep Bilateral Learning for Real-Time Image Enhancementcited by 200
  25. 2016A guide to convolution arithmetic for deep learningcited by 182
  26. 2020SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentationcited by 170
  27. 2020Single-Stage Semantic Segmentation from Image Labelscited by 167
  28. 2021Transformer for Single Image Super-Resolutioncited by 150
  29. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  30. 2019Pixel-Adaptive Convolutional Neural Networkscited by 126
  31. 2019CARAFE: Content-Aware ReAssembly of FEaturescited by 119
  32. 2018Fast End-to-End Trainable Guided Filtercited by 50
  33. 2020Attention-Based Transformers for Instance Segmentation of Cells in Microstructurescited by 38
  34. 2020Learning Affinity-Aware Upsampling for Deep Image Mattingcited by 37
  35. 2022SAPA: Similarity-Aware Point Affiliation for Feature Upsamplingcited by 32
  36. 2022Learning Implicit Feature Alignment Function for Semantic Segmentationcited by 31
  37. 2019Rethinking Softmax with Cross-Entropy: Neural Network Classifier as Mutual Information Estimatorcited by 25
  38. 2020Adaptive confidence thresholding for monocular depth estimationcited by 25

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 30; 2025: 39; 2026: 45.

114 papers.

Sort:

By area:

  1. 2024StableNormal: Reducing Diffusion Variance for Stable and Sharp Normalcited by 153
  2. 2024SAMPart3D: Segment Any Part in 3D Objectscited by 95
  3. 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 91
  4. 2024Improving 2D Feature Representations by 3D-Aware Fine-Tuningcited by 70
  5. 2024AnomalyDINO: Boosting Patch-based Few-shot Anomaly Detection with DINOv2cited by 53
  6. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  7. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 42
  8. 2024MedCLIP-SAMv2: Towards Universal Text-Driven Medical Image Segmentationcited by 37
  9. 2024LiFT: A Surprisingly Simple Lightweight Feature Transform for Dense ViT Descriptorscited by 27
  10. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 26
  11. 2025AnyUp: Universal Feature Upsamplingcited by 26
  12. 2024SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Imagescited by 25
38 more
  1. 2025Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentationcited by 23
  2. 2025LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Modelscited by 23
  3. 2025JAFAR: Jack up Any Feature at Any Resolutioncited by 21
  4. 2024PixOOD: Pixel-Level Out-of-Distribution Detectioncited by 20
  5. 2024Real-Time 4K Super-Resolution of Compressed AVIF Images. AIS 2024 Challenge Surveycited by 19
  6. 2024Keypoint Abstraction using Large Models for Object-Relative Imitation Learningcited by 18
  7. 2025Eval3D: Interpretable and Fine-grained Evaluation for 3D Generationcited by 15
  8. 2024SAM-LAD: Segment Anything Model Meets Zero-Shot Logic Anomaly Detectioncited by 14
  9. 2024T-3DGS: Removing Transient Objects for 3D Scene Reconstructioncited by 13
  10. 2025SteerX: Creating Any Camera-Free 3D and 4D Scenes with Geometric Steeringcited by 13
  11. 2025SkySplat: Generalizable 3D Gaussian Splatting from Multi-Temporal Sparse Satellite Imagescited by 13
  12. 2024Zero-Shot Video Semantic Segmentation based on Pre-Trained Diffusion Modelscited by 12
  13. 2024Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledgecited by 12
  14. 2026INSID3: Training-Free In-Context Segmentation with DINOv3cited by 12
  15. 2025Feed-Forward SceneDINO for Unsupervised Semantic Scene Completioncited by 11
  16. 2024TIGER: Text-Instructed 3D Gaussian Retrieval and Coherent Editingcited by 10
  17. 2024DV-Matcher: Deformation-based Non-Rigid Point Cloud Matching Guided by Pre-trained Visual Featurescited by 10
  18. 2025Online Language Splattingcited by 10
  19. 2025AGO: Adaptive Grounding for Open World 3D Occupancy Predictioncited by 9
  20. 2024LLaVA-UHD v2: an MLLM Integrating High-Resolution Semantic Pyramid via Hierarchical Window Transformercited by 8
  21. 2024A Refreshed Similarity-based Upsampler for Direct High-Ratio Feature Upsamplingcited by 7
  22. 2025Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Imagescited by 7
  23. 2024Upsampling DINOv2 features for unsupervised vision tasks and weakly supervised materials segmentationcited by 6
  24. 2025FeatSharp: Your Vision Model Features, Sharpercited by 6
  25. 2025SAB3R: Semantic-Augmented Backbone in 3D Reconstructioncited by 6
  26. 2025Splat Feature Solvercited by 6
  27. 2025Upsample Anything: A Simple and Hard to Beat Baseline for Feature Upsamplingcited by 6
  28. 2025NAF: Zero-Shot Feature Upsampling via Neighborhood Attention Filteringcited by 6
  29. 2026ConsID-Gen: View-Consistent and Identity-Preserving Image-to-Video Generationcited by 6
  30. 2026LLMind: Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Modelscited by 6
  31. 2025A Survey on Foundation-Model-Based Industrial Defect Detectioncited by 5
  32. 2025CAP-Net: A Unified Network for 6D Pose and Size Estimation of Categorical Articulated Parts from a Single RGB-D Imagecited by 5
  33. 2025DiffEye: Diffusion-Based Continuous Eye-Tracking Data Generation Conditioned on Natural Imagescited by 5
  34. 2025VectorSynth: Fine-Grained Satellite Image Synthesis with Structured Semanticscited by 5
  35. 2026Driving with DINO: Vision Foundation Features as a Unified Bridge for Sim-to-Real Generation in Autonomous Drivingcited by 5
  36. 2024Fast and Efficient: Mask Neural Fields for 3D Scene Segmentationcited by 4
  37. 2024Dynamic Scene Understanding through Object-Centric Voxelization and Neural Renderingcited by 4
  38. 2024Cauchy activation function and XNetcited by 4

50 most cited of 114 All 114 (JSON)