# Minimalist Vision with Freeform Pixels

arXiv: https://arxiv.org/abs/2501.00142 · HTML: https://mlsanity.com/paper/2501.00142/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 30 Dec 2024.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [Deep Optics for Monocular Depth Estimation and 3D Object Detection](https://mlsanity.com/paper/1904.08601/) (2019)
- [Deep Optics for Single-shot High-dynamic-range Imaging](https://mlsanity.com/paper/1908.00620/) (2019)
- [DiffuserCam: Lensless Single-exposure 3D Imaging](https://mlsanity.com/paper/1710.02134/) (2017)
- [The Differentiable Lens: Compound Lens Search over Glass Surfaces and Materials for Object Detection](https://mlsanity.com/paper/2212.04441/) (2022)
- [ASP Vision: Optically Computing the First Layer of Convolutional Neural Networks using Angle Sensitive Pixels](https://mlsanity.com/paper/1605.03621/) (2016)
- [Image Super-Resolution Using T-Tetromino Pixels](https://mlsanity.com/paper/2111.09013/) (2021)

## Built on by (most cited first, 5 of 5)

- [Direct Kernel Optimization: Efficient Design for Opto-Electronic Convolutional Neural Networks](https://mlsanity.com/paper/2511.02065/) (2025)
- [Computer Vision with a Superpixelation Camera](https://mlsanity.com/paper/2603.26900/) (2026)
- [Imaging Hidden Objects with Consumer LiDAR via Motion Induced Sampling](https://mlsanity.com/paper/2605.17865/) (2026)
- [Low Latency Gaze Tracking via Latent Optical Sensing](https://mlsanity.com/paper/2605.17990/) (2026)
- [Minimalist Visual Inertial Odometry](https://mlsanity.com/paper/2605.19990/) (2026)

## Abstract (arXiv)

A minimalist vision system uses the smallest number of pixels needed to solve a vision task. While traditional cameras use a large grid of square pixels, a minimalist camera uses freeform pixels that can take on arbitrary shapes to increase their information content. We show that the hardware of a minimalist camera can be modeled as the first layer of a neural network, where the subsequent layers are used for inference. Training the network for any given task yields the shapes of the camera's freeform pixels, each of which is implemented using a photodetector and an optical mask. We have designed minimalist cameras for monitoring indoor spaces (with 8 pixels), measuring room lighting (with 8 pixels), and estimating traffic flow (with 8 pixels). The performance demonstrated by these systems is on par with a traditional camera with orders of magnitude more pixels. Minimalist vision has two major advantages. First, it naturally tends to preserve the privacy of individuals in the scene since the captured information is inadequate for extracting visual details. Second, since the number of measurements made by a minimalist camera is very small, we show that it can be fully self-powered, i.e., function without an external power supply or a battery.