# MetaMorphosis: Task-oriented Privacy Cognizant Feature Generation for Multi-task Learning

arXiv: https://arxiv.org/abs/2305.07815 · HTML: https://mlsanity.com/paper/2305.07815/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 49 references that are in the map.
- First posted on arXiv 13 May 2023.

## Builds on (most cited first, 30 of 36)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [ResNeSt: Split-Attention Networks](https://mlsanity.com/paper/2004.08955/) (2020)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [Opacus: User-Friendly Differential Privacy Library in PyTorch](https://mlsanity.com/paper/2109.12298/) (2021)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [Structured Knowledge Distillation for Dense Prediction](https://mlsanity.com/paper/1903.04197/) (2019)
- [PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization](https://mlsanity.com/paper/1905.13727/) (2019)
- [Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos](https://mlsanity.com/paper/1906.06876/) (2019)
- [Deep Learning with Label Differential Privacy](https://mlsanity.com/paper/2102.06062/) (2021)
- [Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis](https://mlsanity.com/paper/2011.06961/) (2020)
- [On the Integration of Self-Attention and Convolution](https://mlsanity.com/paper/2111.14556/) (2021)
- [Universal Representation Learning from Multiple Domains for Few-shot Classification](https://mlsanity.com/paper/2103.13841/) (2021)
- [SA-Net: Shuffle Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/2102.00240/) (2021)
- [Knowledge Distillation for Multi-task Learning](https://mlsanity.com/paper/2007.06889/) (2020)
- [LEDNet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1905.02423/) (2019)
- [Deep feature compression for collaborative object detection](https://mlsanity.com/paper/1802.03931/) (2018)
- [Adversarial Learning of Privacy-Preserving and Task-Oriented Representations](https://mlsanity.com/paper/1911.10143/) (2019)
- [DenseMTL: Cross-task Attention Mechanism for Dense Multi-task Learning](https://mlsanity.com/paper/2206.08927/) (2022)
- [Cross-Dataset Collaborative Learning for Semantic Segmentation in Autonomous Driving](https://mlsanity.com/paper/2103.11351/) (2021)
- [Universal Representations: A Unified Look at Multiple Task and Domain Learning](https://mlsanity.com/paper/2204.02744/) (2022)

## Abstract (arXiv)

With the growth of computer vision applications, deep learning, and edge computing contribute to ensuring practical collaborative intelligence (CI) by distributing the workload among edge devices and the cloud. However, running separate single-task models on edge devices is inefficient regarding the required computational resource and time. In this context, multi-task learning allows leveraging a single deep learning model for performing multiple tasks, such as semantic segmentation and depth estimation on incoming video frames. This single processing pipeline generates common deep features that are shared among multi-task modules. However, in a collaborative intelligence scenario, generating common deep features has two major issues. First, the deep features may inadvertently contain input information exposed to the downstream modules (violating input privacy). Second, the generated universal features expose a piece of collective information than what is intended for a certain task, in which features for one task can be utilized to perform another task (violating task privacy). This paper proposes a novel deep learning-based privacy-cognizant feature generation process called MetaMorphosis that limits inference capability to specific tasks at hand. To achieve this, we propose a channel squeeze-excitation based feature metamorphosis module, Cross-SEC, to achieve distinct attention of all tasks and a de-correlation loss function with differential-privacy to train a deep learning model that produces distinct privacy-aware features as an output for the respective tasks. With extensive experimentation on four datasets consisting of diverse images related to scene understanding and facial attributes, we show that MetaMorphosis outperforms recent adversarial learning and universal feature generation methods by guaranteeing privacy requirements in an efficient way for image and video analytics.