# Resource Efficient Perception for Vision Systems

arXiv: https://arxiv.org/abs/2405.07166 · HTML: https://mlsanity.com/paper/2405.07166/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 47 references that are in the map.
- First posted on arXiv 12 May 2024.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [UnitBox: An Advanced Object Detection Network](https://mlsanity.com/paper/1608.01471/) (2016)
- [Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges](https://mlsanity.com/paper/2102.12219/) (2021)
- [Towards Large-Scale Small Object Detection: Survey and Benchmarks](https://mlsanity.com/paper/2207.14096/) (2022)
- [QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection](https://mlsanity.com/paper/2103.09136/) (2021)
- [Differentiable Patch Selection for Image Recognition](https://mlsanity.com/paper/2104.03059/) (2021)
- [Collaborative Global-Local Networks for Memory-Efficient Segmentation of Ultra-High Resolution Images](https://mlsanity.com/paper/1905.06368/) (2019)
- [Streaming convolutional neural networks for end-to-end learning with multi-megapixel images](https://mlsanity.com/paper/1911.04432/) (2019)
- [On the adequacy of untuned warmup for adaptive optimization](https://mlsanity.com/paper/1910.04209/) (2019)
- [Progressive Semantic Segmentation](https://mlsanity.com/paper/2104.03778/) (2021)
- [TJU-DHD: A Diverse High-Resolution Dataset for Object Detection](https://mlsanity.com/paper/2011.09170/) (2020)
- [High Quality Segmentation for Ultra High-resolution Images](https://mlsanity.com/paper/2111.14482/) (2021)
- [Processing Megapixel Images with Deep Attention-Sampling Models](https://mlsanity.com/paper/1905.03711/) (2019)
- [Ultra-High Resolution Segmentation with Ultra-Rich Context: A Novel Benchmark](https://mlsanity.com/paper/2305.10899/) (2023)
- [Hard-Attention for Scalable Image Classification](https://mlsanity.com/paper/2102.10212/) (2021)
- [Efficient Classification of Very Large Images with Tiny Objects](https://mlsanity.com/paper/2106.02694/) (2021)
- [A Full-Image Full-Resolution End-to-End-Trainable CNN Framework for Image Forgery Detection](https://mlsanity.com/paper/1909.06751/) (2019)
- [Iterative Patch Selection for High-Resolution Image Recognition](https://mlsanity.com/paper/2210.13007/) (2022)
- [AdaZoom: Adaptive Zoom Network for Multi-Scale Object Detection in Large Scenes](https://mlsanity.com/paper/2106.10409/) (2021)

## Abstract (arXiv)

Despite the rapid advancement in the field of image recognition, the processing of high-resolution imagery remains a computational challenge. However, this processing is pivotal for extracting detailed object insights in areas ranging from autonomous vehicle navigation to medical imaging analyses. Our study introduces a framework aimed at mitigating these challenges by leveraging memory efficient patch based processing for high resolution images. It incorporates a global context representation alongside local patch information, enabling a comprehensive understanding of the image content. In contrast to traditional training methods which are limited by memory constraints, our method enables training of ultra high resolution images. We demonstrate the effectiveness of our method through superior performance on 7 different benchmarks across classification, object detection, and segmentation. Notably, the proposed method achieves strong performance even on resource-constrained devices like Jetson Nano. Our code is available at https://github.com/Visual-Conception-Group/Localized-Perception-Constrained-Vision-Systems.