mlsanityData to 7 Oct 2026

Deformably-Scaled Transposed Convolution

arXiv 2210.09446 ↗First posted 17 Oct 2022cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Transposed convolution is crucial for generating high-resolution outputs, yet has received little attention compared to convolution layers. In this work we revisit transposed convolution and introduce a novel layer that allows us to place information in the image selectively and choose the `stroke breadth' at which the image is synthesized, whilst incurring a small additional parameter cost. For this we introduce three ideas: firstly, we regress offsets to the positions where the transpose convolution results are placed; secondly we broadcast the offset weight locations over a learnable neighborhood; and thirdly we use a compact parametrization to share weights and restrict offsets. We show that simply substituting upsampling operators with our novel layer produces substantial improvements across tasks as diverse as instance segmentation, object detection, semantic segmentation, generative image modeling, and 3D magnetic resonance image enhancement, while outperforming all existing variants of transposed convolutions. Our novel layer can be used as a drop-in replacement for 2D and 3D upsampling operators and the code will be publicly available.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 52 references that are in the map. First posted on arXiv 17 Oct 2022.

Builds on

40 of 52 references.

References · 40

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2017Mask R-CNNcited by 7,259
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  9. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  10. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  11. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  12. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
28 more
  1. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  2. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  3. 2016Pyramid Scene Parsing Networkcited by 2,472
  4. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  5. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  6. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  7. 2017Deformable Convolutional Networkscited by 1,481
  8. 2015Spatial Transformer Networkscited by 1,423
  9. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  10. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  11. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  12. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  13. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  14. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 666
  15. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  16. 2017Dilated Residual Networkscited by 293
  17. 2017ICNet for Real-Time Semantic Segmentation on High-Resolution Imagescited by 261
  18. 2017The Reversible Residual Network: Backpropagation Without Storing Activationscited by 215
  19. 2019Pixel-Adaptive Convolutional Neural Networkscited by 126
  20. 2019CARAFE: Content-Aware ReAssembly of FEaturescited by 119
  21. 2020Delving Deeper into Anti-aliasing in ConvNetscited by 60
  22. 2019Degenerative Adversarial NeuroImage Nets for Brain Scan Simulations: Application in Ageing and Dementiacited by 34
  23. 2017Active Convolution: Learning the Shape of Convolution for Image Classificationcited by 24
  24. 2017The Devil is in the Decoder: Classification, Regression and GANscited by 22
  25. 2018Autofocus Layer for Semantic Segmentationcited by 13
  26. 2018Deeper Image Quality Transfer: Training Low-Memory Neural Networks for 3D Imagescited by 8
  27. 2019Deep Learning for Low-Field to High-Field MR: Image Quality Transfer with Probabilistic Decimation Simulatorcited by 8
  28. 2019Multi-Stage Prediction Networks for Data Harmonizationcited by 7

Built on by

Not cited yet.