mlsanityData to 14 Oct 2026

Learning to Zoom and Unzoom

arXiv 2303.15390 ↗First posted 27 Mar 2023cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Many perception systems in mobile computing, autonomous navigation, and AR/VR face strict compute constraints that are particularly challenging for high-resolution input images. Previous works propose nonuniform downsamplers that "learn to zoom" on salient image regions, reducing compute while retaining task-relevant image information. However, for tasks with spatial labels (such as 2D/3D object detection and semantic segmentation), such distortions may harm performance. In this work (LZU), we "learn to zoom" in on the input image, compute spatial features, and then "unzoom" to revert any deformations. To enable efficient and differentiable unzooming, we approximate the zooming warp with a piecewise bilinear mapping that is invertible. LZU can be applied to any task with 2D spatial input and any model with 2D spatial features, and we demonstrate this versatility by evaluating on a variety of tasks and datasets: object detection on Argoverse-HD, semantic segmentation on Cityscapes, and monocular 3D object detection on nuScenes. Interestingly, we observe boosts in performance even when high-resolution sensor data is unavailable, implying that LZU can be used to "learn to upsample" as well.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 23 of its 29 references that are in the map. First posted on arXiv 27 Mar 2023.

Builds on

23 of 29 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  7. 2016Pyramid Scene Parsing Networkcited by 2,479
  8. 2017Deformable Convolutional Networkscited by 1,488
  9. 2015Spatial Transformer Networkscited by 1,429
  10. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  11. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 670
  12. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
11 more
  1. 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 301
  2. 2019Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inferencecited by 91
  3. 2018Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networkscited by 56
  4. 2020Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolationcited by 51
  5. 2019Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraintscited by 37
  6. 2019Efficient Segmentation: Learning Downsampling Near Semantic Boundariescited by 33
  7. 2021FOVEA: Foveated Image Magnification for Autonomous Navigationcited by 32
  8. 2021Learning to Downsample for Segmentation of Ultra-High Resolution Imagescited by 19
  9. 2018Pixel-wise Attentional Gating for Parsimonious Pixel Labelingcited by 12
  10. 2020SegBlocks: Block-Based Dynamic Resolution Networks for Real-Time Segmentationcited by 9
  11. 2017Dense Transformer Networkscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 2; 2026: 5.

11 papers.

  1. 2023ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classificationcited by 33
  2. 2023ZoomTrack: Target-aware Non-uniform Resizing for Efficient Visual Trackingcited by 25
  3. 2024Instance-Warp: Saliency Guided Image Warping for Unsupervised Domain Adaptationcited by 7
  4. 2025Spatial Frequency Modulation for Semantic Segmentationcited by 3
  5. 2026Adaptive Image Zoom-in with Bounding Box Transformation for UAV Object Detectioncited by 2
  6. 2024VPOcc: Exploiting Vanishing Point for 3D Semantic Occupancy Predictioncited by 1
  7. 2025Foveated Instance Segmentationcited by 1
  8. 2026UHR-DETR: Efficient End-to-End Small Object Detection for Ultra-High-Resolution Remote Sensing Imagerycited by 0
  9. 2026WarpI2I: Image Warping for Image-to-Image Translationcited by 0
  10. 2026Variable-Granularity Tokenization for High-Resolution Object Detectioncited by 0
  11. 2026The Earth in One Gaze: Training-Free Active Focus for UHR Remote Sensing Understandingcited by 0