# Rubik's Optical Neural Networks: Multi-task Learning with Physics-aware Rotation Architecture

arXiv: https://arxiv.org/abs/2304.12985 · HTML: https://mlsanity.com/paper/2304.12985/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 25 Apr 2023.

## Builds on (most cited first, 15 of 15)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [Neuromorphic Silicon Photonic Networks](https://mlsanity.com/paper/1611.02272/) (2016)
- [An Evaluation of Edge TPU Accelerators for Convolutional Neural Networks](https://mlsanity.com/paper/2102.10423/) (2021)
- [Ensemble learning of diffractive optical networks](https://mlsanity.com/paper/2009.06869/) (2020)
- [Snapshot Multispectral Imaging Using a Diffractive Optical Network](https://mlsanity.com/paper/2212.05217/) (2022)
- [Scale-, shift- and rotation-invariant diffractive optical networks](https://mlsanity.com/paper/2010.12747/) (2020)
- [RESPECT: Reinforcement Learning based Edge Scheduling on Pipelined Coral Edge TPUs](https://mlsanity.com/paper/2304.04716/) (2023)
- [All-optical graph representation learning using integrated diffractive photonic computing units](https://mlsanity.com/paper/2204.10978/) (2022)
- [Physics-aware Differentiable Discrete Codesign for Diffractive Optical Neural Networks](https://mlsanity.com/paper/2209.14252/) (2022)
- [Optical multi-task learning using multi-wavelength diffractive deep neural networks](https://mlsanity.com/paper/2212.00022/) (2022)
- [Real-time Multi-Task Diffractive Deep Neural Networks via Hardware-Software Co-design](https://mlsanity.com/paper/2012.08906/) (2020)

## Built on by (most cited first, 3 of 3)

- [SP2RINT: Spatially-Decoupled Physics-Inspired Progressive Inverse Optimization for Scalable, PDE-Constrained Meta-Optical Neural Network Training](https://mlsanity.com/paper/2505.18377/) (2025)
- [Illumination Angular Spectrum Encoding for Controlling the Functionality of Diffractive Networks](https://mlsanity.com/paper/2601.04825/) (2026)
- [ReDON: Recurrent Diffractive Optical Neural Processor with Reconfigurable Self-Modulated Nonlinearity](https://mlsanity.com/paper/2602.23616/) (2026)

## Abstract (arXiv)

Recently, there are increasing efforts on advancing optical neural networks (ONNs), which bring significant advantages for machine learning (ML) in terms of power efficiency, parallelism, and computational speed. With the considerable benefits in computation speed and energy efficiency, there are significant interests in leveraging ONNs into medical sensing, security screening, drug detection, and autonomous driving. However, due to the challenge of implementing reconfigurability, deploying multi-task learning (MTL) algorithms on ONNs requires re-building and duplicating the physical diffractive systems, which significantly degrades the energy and cost efficiency in practical application scenarios. This work presents a novel ONNs architecture, namely, textitRubikONNs, which utilizes the physical properties of optical systems to encode multiple feed-forward functions by physically rotating the hardware similarly to rotating a textitRubik's Cube. To optimize MTL performance on RubikONNs, two domain-specific physics-aware training algorithms textitRotAgg and textitRotSeq are proposed. Our experimental results demonstrate more than 4× improvements in energy and cost efficiency with marginal accuracy degradation compared to the state-of-the-art approaches.