# PixelNet: Representation of the pixels, by the pixels, and for the pixels

arXiv: https://arxiv.org/abs/1702.06506 · HTML: https://mlsanity.com/paper/1702.06506/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2017.

## Built on by (most cited first, 17 of 17)

- [Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenes](https://mlsanity.com/paper/2104.06935/) (2021)
- [KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypoints](https://mlsanity.com/paper/2205.04992/) (2022)
- [Probabilistic Graph Attention Network with Conditional Kernels for Pixel-Wise Prediction](https://mlsanity.com/paper/2101.02843/) (2021)
- [VideoClick: Video Object Segmentation with a Single Click](https://mlsanity.com/paper/2101.06545/) (2021)
- [DivergentNets: Medical Image Segmentation by Network Ensemble](https://mlsanity.com/paper/2107.00283/) (2021)
- [Multi-Task Learning for Visual Scene Understanding](https://mlsanity.com/paper/2203.14896/) (2022)
- [Streaming Self-Training via Domain-Agnostic Unlabeled Images](https://mlsanity.com/paper/2104.03309/) (2021)
- [Road-aware Monocular Structure from Motion and Homography Estimation](https://mlsanity.com/paper/2112.08635/) (2021)
- [Urban land-use analysis using proximate sensing imagery: a survey](https://mlsanity.com/paper/2101.04827/) (2021)
- [Variational Structured Attention Networks for Deep Visual Representation Learning](https://mlsanity.com/paper/2103.03510/) (2021)
- [Class Balanced PixelNet for Neurological Image Segmentation](https://mlsanity.com/paper/2204.11048/) (2022)
- [SGW-based Multi-Task Learning in Vision Tasks](https://mlsanity.com/paper/2410.03778/) (2024)
- [Video-Specific Autoencoders for Exploring, Editing and Transmitting Videos](https://mlsanity.com/paper/2103.17261/) (2021)
- [Glioma Prognosis: Segmentation of the Tumor and Survival Prediction using Shape, Geometric and Clinical Information](https://mlsanity.com/paper/2104.00980/) (2021)
- [Medusa: Universal Feature Learning via Attentional Multitasking](https://mlsanity.com/paper/2204.05698/) (2022)
- [MLMT-CNN for Object Detection and Segmentation in Multi-layer and Multi-spectral Images](https://mlsanity.com/paper/2407.14473/) (2024)
- [Ischemic Stroke Lesion Segmentation Using Adversarial Learning](https://mlsanity.com/paper/2204.04993/) (2022)

## Abstract (arXiv)

We explore design principles for general pixel-level prediction problems, from low-level edge detection to mid-level surface normal estimation to high-level semantic segmentation. Convolutional predictors, such as the fully-convolutional network (FCN), have achieved remarkable success by exploiting the spatial redundancy of neighboring pixels through convolutional processing. Though computationally efficient, we point out that such approaches are not statistically efficient during learning precisely because spatial redundancy limits the information learned from neighboring pixels. We demonstrate that stratified sampling of pixels allows one to (1) add diversity during batch updates, speeding up learning; (2) explore complex nonlinear predictors, improving accuracy; and (3) efficiently train state-of-the-art models tabula rasa (i.e., "from scratch") for diverse pixel-labeling tasks. Our single architecture produces state-of-the-art results for semantic segmentation on PASCAL-Context dataset, surface normal estimation on NYUDv2 depth dataset, and edge detection on BSDS.