# Rethinking Encoder-Decoder Flow Through Shared Structures

arXiv: https://arxiv.org/abs/2501.14535 · HTML: https://mlsanity.com/paper/2501.14535/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 31 references that are in the map.
- First posted on arXiv 24 Jan 2025.

## Builds on (most cited first, 26 of 26)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [EVA: Exploring the Limits of Masked Visual Representation Learning at Scale](https://mlsanity.com/paper/2211.07636/) (2022)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation](https://mlsanity.com/paper/2312.02145/) (2023)
- [MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation](https://mlsanity.com/paper/2307.14460/) (2023)
- [FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization](https://mlsanity.com/paper/2303.14189/) (2023)
- [RepViT: Revisiting Mobile CNN From ViT Perspective](https://mlsanity.com/paper/2307.09283/) (2023)
- [Learning the Depths of Moving People by Watching Frozen People](https://mlsanity.com/paper/1904.11111/) (2019)
- [Fast Vision Transformers with HiLo Attention](https://mlsanity.com/paper/2205.13213/) (2022)
- [DepthFM: Fast Monocular Depth Estimation with Flow Matching](https://mlsanity.com/paper/2403.13788/) (2024)
- [SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications](https://mlsanity.com/paper/2303.15446/) (2023)
- [Learning to Upsample by Learning to Sample](https://mlsanity.com/paper/2308.15085/) (2023)
- [SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design](https://mlsanity.com/paper/2401.16456/) (2024)
- [Denoising Vision Transformers](https://mlsanity.com/paper/2401.02957/) (2024)
- [More than Encoder: Introducing Transformer Decoder to Upsample](https://mlsanity.com/paper/2106.10637/) (2021)
- [Real-time Monocular Depth Estimation with Sparse Supervision on Mobile](https://mlsanity.com/paper/2105.12053/) (2021)

## Abstract (arXiv)

Dense prediction tasks have enjoyed a growing complexity of encoder architectures, decoders, however, have remained largely the same. They rely on individual blocks decoding intermediate feature maps sequentially. We introduce banks, shared structures that are used by each decoding block to provide additional context in the decoding process. These structures, through applying them via resampling and feature fusion, improve performance on depth estimation for state-of-the-art transformer-based architectures on natural and synthetic images whilst training on large-scale datasets.