# Channel-Aware Distillation Transformer for Depth Estimation on Nano Drones

arXiv: https://arxiv.org/abs/2303.10386 · HTML: https://mlsanity.com/paper/2303.10386/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 56 references that are in the map.
- First posted on arXiv 18 Mar 2023.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [On the Efficacy of Knowledge Distillation](https://mlsanity.com/paper/1910.01348/) (2019)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry](https://mlsanity.com/paper/2003.01060/) (2020)
- [Does Knowledge Distillation Really Work?](https://mlsanity.com/paper/2106.05945/) (2021)
- [On the uncertainty of self-supervised monocular depth estimation](https://mlsanity.com/paper/2005.06209/) (2020)
- [Channel-wise Knowledge Distillation for Dense Prediction](https://mlsanity.com/paper/2011.13256/) (2020)
- [HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2012.07356/) (2020)
- [Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2211.13202/) (2022)
- [FastDepth: Fast Monocular Depth Estimation on Embedded Systems](https://mlsanity.com/paper/1903.03273/) (2019)
- [Knowledge Distillation via the Target-aware Transformer](https://mlsanity.com/paper/2205.10793/) (2022)
- [Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2112.13047/) (2021)
- [Fine-grained Semantics-aware Representation Enhancement for Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2108.08829/) (2021)
- [Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimation](https://mlsanity.com/paper/1903.04202/) (2019)
- [Learning to Fly by Crashing](https://mlsanity.com/paper/1704.05588/) (2017)
- [Deep Digging into the Generalization of Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2205.11083/) (2022)
- [A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones](https://mlsanity.com/paper/1805.01831/) (2018)
- [Channel Distillation: Channel-Wise Attention for Knowledge Distillation](https://mlsanity.com/paper/2006.01683/) (2020)
- [Boosting Light-Weight Depth Estimation Via Knowledge Distillation](https://mlsanity.com/paper/2105.06143/) (2021)
- [Sniffy Bug: A Fully Autonomous Swarm of Gas-Seeking Nano Quadcopters in Cluttered Environments](https://mlsanity.com/paper/2107.05490/) (2021)
- [Efficient Optical flow and Stereo Vision for Velocity Estimation and Obstacle Avoidance on an Autonomous Pocket Drone](https://mlsanity.com/paper/1612.06702/) (2016)
- [NanoFlowNet: Real-time Dense Optical Flow on a Nano Quadcopter](https://mlsanity.com/paper/2209.06918/) (2022)

## Built on by (most cited first, 1 of 1)

- [Predicting High-precision Depth on Low-Precision Devices Using 2D Hilbert Curves](https://mlsanity.com/paper/2405.14024/) (2024)

## Abstract (arXiv)

Autonomous navigation of drones using computer vision has achieved promising performance. Nano-sized drones based on edge computing platforms are lightweight, flexible, and cheap, thus suitable for exploring narrow spaces. However, due to their extremely limited computing power and storage, vision algorithms designed for high-performance GPU platforms cannot be used for nano drones. To address this issue this paper presents a lightweight CNN depth estimation network deployed on nano drones for obstacle avoidance. Inspired by Knowledge Distillation (KD), a Channel-Aware Distillation Transformer (CADiT) is proposed to facilitate the small network to learn knowledge from a larger network. The proposed method is validated on the KITTI dataset and tested on a nano drone Crazyflie, with an ultra-low power microprocessor GAP8.