# Distilling Knowledge for Designing Computational Imaging Systems

arXiv: https://arxiv.org/abs/2501.17898 · HTML: https://mlsanity.com/paper/2501.17898/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 72 references that are in the map.
- First posted on arXiv 29 Jan 2025.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [DOLCE: A Model-Based Probabilistic Diffusion Framework for Limited-Angle CT Reconstruction](https://mlsanity.com/paper/2211.12340/) (2022)
- [Deep Optics for Single-shot High-dynamic-range Imaging](https://mlsanity.com/paper/1908.00620/) (2019)
- [Knowledge Distillation in Deep Learning and its Applications](https://mlsanity.com/paper/2007.09029/) (2020)
- [Physics-based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging](https://mlsanity.com/paper/1808.03571/) (2018)
- [Deep Coded Aperture Design: An End-to-End Approach for Computational Imaging Tasks](https://mlsanity.com/paper/2105.03390/) (2021)
- [On Retrospective k-space Subsampling schemes For Deep MRI Reconstruction](https://mlsanity.com/paper/2301.08365/) (2023)
- [Optimal MRI Undersampling Patterns for Ultimate Benefit of Medical Vision Tasks](https://mlsanity.com/paper/2108.04914/) (2021)
- [Deep Optical Coding Design in Computational Imaging](https://mlsanity.com/paper/2207.00164/) (2022)
- [Depth Estimation from a Single Optical Encoded Image using a Learned Colored-Coded Aperture](https://mlsanity.com/paper/2309.08033/) (2023)
- [Constrained Probabilistic Mask Learning for Task-specific Undersampled MRI Reconstruction](https://mlsanity.com/paper/2305.16376/) (2023)
- [SnapCap: Efficient Snapshot Compressive Video Captioning](https://mlsanity.com/paper/2401.04903/) (2024)
- [Learning Knowledge Representation with Meta Knowledge Distillation for Single Image Super-Resolution](https://mlsanity.com/paper/2207.08356/) (2022)
- [Highly Constrained Coded Aperture Imaging Systems Design Via a Knowledge Distillation Approach](https://mlsanity.com/paper/2406.17970/) (2024)

## Abstract (arXiv)

Designing the physical encoder is crucial for accurate image reconstruction in computational imaging (CI) systems. Currently, these systems are designed via end-to-end (E2E) optimization, where the encoder is modeled as a neural network layer and is jointly optimized with the decoder. However, the performance of E2E optimization is significantly reduced by the physical constraints imposed on the encoder. Also, since the E2E learns the parameters of the encoder by backpropagating the reconstruction error, it does not promote optimal intermediate outputs and suffers from gradient vanishing. To address these limitations, we reinterpret the concept of knowledge distillation (KD) for designing a physically constrained CI system by transferring the knowledge of a pretrained, less-constrained CI system. Our approach involves three steps: (1) Given the original CI system (student), a teacher system is created by relaxing the constraints on the student's encoder. (2) The teacher is optimized to solve a less-constrained version of the student's problem. (3) The teacher guides the training of the student through two proposed knowledge transfer functions, targeting both the encoder and the decoder feature space. The proposed method can be employed to any imaging modality since the relaxation scheme and the loss functions can be adapted according to the physical acquisition and the employed decoder. This approach was validated on three representative CI modalities: magnetic resonance, single-pixel, and compressive spectral imaging. Simulations show that a teacher system with an encoder that has a structure similar to that of the student encoder provides effective guidance. Our approach achieves significantly improved reconstruction performance and encoder design, outperforming both E2E optimization and traditional non-data-driven encoder designs.