mlsanityData to 7 Oct 2026

EVP: Enhanced Visual Perception using Inverse Multi-Attentive Feature Refinement and Regularized Image-Text Alignment

arXiv 2312.08548 ↗First posted 13 Dec 2023cs.CVcited by 14

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

This work presents the network architecture EVP (Enhanced Visual Perception). EVP builds on the previous work VPD which paved the way to use the Stable Diffusion network for computer vision tasks. We propose two major enhancements. First, we develop the Inverse Multi-Attentive Feature Refinement (IMAFR) module which enhances feature learning capabilities by aggregating spatial information from higher pyramid levels. Second, we propose a novel image-text alignment module for improved feature extraction of the Stable Diffusion backbone. The resulting architecture is suitable for a wide variety of tasks and we demonstrate its performance in the context of single-image depth estimation with a specialized decoder using classification-based bins and referring segmentation with an off-the-shelf decoder. Comprehensive experiments conducted on established datasets show that EVP achieves state-of-the-art results in single-image depth estimation for indoor (NYU Depth v2, 11.8% RMSE improvement over VPD) and outdoor (KITTI) environments, as well as referring segmentation (RefCOCO, 2.53 IoU improvement over ReLA). The code and pre-trained models are publicly available at https://github.com/Lavreniuk/EVP.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 57 of its 61 references that are in the map. First posted on arXiv 13 Dec 2023.

Builds on

57 of 61 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Unleashing Text-to-Image Diffusion Models for Visual Perception3 Mar
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  3. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  8. 2017Squeeze-and-Excitation Networkscited by 4,763
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  11. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  12. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
38 more
  1. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  2. 2021Vision Transformers for Dense Predictioncited by 1,789
  3. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,518
  4. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  5. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  6. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  7. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
  8. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  9. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  10. 2019UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentationcited by 583
  11. 2019ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networkscited by 572
  12. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  13. 2019Selective Kernel Networkscited by 401
  14. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  15. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  16. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  17. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  18. 2020Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentationcited by 204
  19. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  20. 2022BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimationcited by 166
  21. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  22. 2020Referring Image Segmentation via Cross-Modal Progressive Comprehensioncited by 131
  23. 2022Revealing the Dark Secrets of Masked Image Modelingcited by 117
  24. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112
  25. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  26. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
  27. 2022P3Depth: Monocular Depth Estimation with a Piecewise Planarity Priorcited by 106
  28. 2022LocalBins: Improving Depth Estimation by Learning Local Distributionscited by 92
  29. 2022Attention Attention Everywhere: Monocular Depth Prediction with Skip Attentioncited by 92
  30. 2023Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animatorcited by 74
  31. 2023iDisc: Internal Discretization for Monocular Depth Estimationcited by 71
  32. 2023All in Tokens: Unifying Output Space of Visual Tasks via Soft Tokencited by 52
  33. 2023NDDepth: Normal-Distance Assisted Monocular Depth Estimationcited by 45
  34. 2023GEDepth: Ground Embedding for Monocular Depth Estimationcited by 26
  35. 2023SQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimationcited by 21
  36. 2023PolyMaX: General Dense Prediction with Mask Transformercited by 14
  37. 2019CU-Net: Cascaded U-Net with Loss Weighted Sampling for Brain Tumor Segmentationcited by 6
  38. 2023MeSa: Masked, Geometric, and Supervised Pre-training for Monocular Depth Estimationcited by 4

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 5.

14 papers.

Sort:

By area:

  1. 2024PrimeDepth: Efficient Monocular Depth Estimation with a Stable Diffusion Preimagecited by 11
  2. 2025DiffRIS: Enhancing Referring Remote Sensing Image Segmentation with Pre-trained Text-to-Image Diffusion Modelscited by 6
  3. 2025Delineate Anything: Resolution-Agnostic Field Boundary Delineation on Satellite Imagerycited by 5
  4. 2024Depth Estimation Based on 3D Gaussian Splatting Siamese Defocuscited by 3
  5. 2025MINGLE: VLMs for Semantically Complex Region Detection in Urban Scenescited by 3
  6. 2025Towards Depth Foundation Model: Recent Trends in Vision-Based Depth Estimationcited by 2
  7. 2024Unlocking Comics: The AI4VA Dataset for Visual Understandingcited by 1
  8. 2024Multi-Modality Driven LoRA for Adverse Condition Depth Estimationcited by 0
  9. 2025VGLD: Visually-Guided Linguistic Disambiguation for Monocular Depth Scale Recoverycited by 0
  10. 2026Hierarchical Awareness Adapters with Hybrid Pyramid Feature Fusion for Dense Depth Predictioncited by 0
  11. 2026Monocular Depth Estimation From the Perspective of Feature Restoration: A Diffusion Enhanced Depth Restoration Approachcited by 0
  12. 2026Delineate Anything v2: A Global Foundation Model for Field Delineationcited by 0
2 more
  1. 2026Beyond Visual Ambiguity: Guiding Robust Monocular Depth Estimation in Challenging Scenarios via Detailed Long Captionscited by 0
  2. 2026A Vision-Language Framework for Measuring Social Life on Sidewalkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Visual Bridge: Universal Visual Perception Representations Generating
  2. 2023DDP: Diffusion Model for Dense Visual Prediction
  3. 2025Parameter-Inverted Image Pyramid Networks for Visual Perception and Multimodal Understanding
  4. 2026EagleDepth: Efficient Fine-Grained Depth Estimation via Pixel Diffusion Decoder
  5. 2025Leveraging Stable Diffusion for Monocular Depth Estimation via Image Semantic Encoding
  6. 2024Diff-2-in-1: Bridging Generation and Dense Perception with Diffusion Models
  7. 2026EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception
  8. 2025Perception Encoder: The best visual embeddings are not at the output of the network