# Deep Joint Source-Channel Coding for Multi-Task Network

arXiv: https://arxiv.org/abs/2109.05779 · HTML: https://mlsanity.com/paper/2109.05779/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 13 Sep 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Density Modeling of Images using a Generalized Normalization Transformation](https://mlsanity.com/paper/1511.06281/) (2015)
- [BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systems](https://mlsanity.com/paper/1910.14315/) (2019)
- [Deep feature compression for collaborative object detection](https://mlsanity.com/paper/1802.03931/) (2018)
- [FSSD: Feature Fusion Single Shot Multibox Detector](https://mlsanity.com/paper/1712.00960/) (2017)
- [BlitzNet: A Real-Time Deep Network for Scene Understanding](https://mlsanity.com/paper/1708.02813/) (2017)
- [Multi-task learning with compressible features for Collaborative Intelligence](https://mlsanity.com/paper/1902.05179/) (2019)
- [Near-Lossless Deep Feature Compression for Collaborative Intelligence](https://mlsanity.com/paper/1804.09963/) (2018)
- [Bit Allocation for Multi-Task Collaborative Intelligence](https://mlsanity.com/paper/2002.07048/) (2020)
- [Deep Joint Source-Channel Coding for Wireless Image Retrieval](https://mlsanity.com/paper/1910.12703/) (2019)
- [Driving Scene Perception Network: Real-time Joint Detection, Depth Estimation and Semantic Segmentation](https://mlsanity.com/paper/1803.03778/) (2018)
- [Triply Supervised Decoder Networks for Joint Detection and Segmentation](https://mlsanity.com/paper/1809.09299/) (2018)
- [Joint Device-Edge Inference over Wireless Links with Pruning](https://mlsanity.com/paper/2003.02027/) (2020)

## Built on by (most cited first, 5 of 5)

- [VideoQA-SC: Adaptive Semantic Communication for Video Question Answering](https://mlsanity.com/paper/2406.18538/) (2024)
- [Multi-Modal Self-Supervised Semantic Communication](https://mlsanity.com/paper/2503.13940/) (2025)
- [MetaMorphosis: Task-oriented Privacy Cognizant Feature Generation for Multi-task Learning](https://mlsanity.com/paper/2305.07815/) (2023)
- [sDAC -- Semantic Digital Analog Converter for Semantic Communications](https://mlsanity.com/paper/2405.02335/) (2024)
- [FRESHLATENT: Channel-Aware Latent Adaptation for Resource-Constrained Embodied VLM Perception](https://mlsanity.com/paper/2609.30629/) (2026)

## Abstract (arXiv)

Multi-task learning (MTL) is an efficient way to improve the performance of related tasks by sharing knowledge. However, most existing MTL networks run on a single end and are not suitable for collaborative intelligence (CI) scenarios. In this work, we propose an MTL network with a deep joint source-channel coding (JSCC) framework, which allows operating under CI scenarios. We first propose a feature fusion based MTL network (FFMNet) for joint object detection and semantic segmentation. Compared with other MTL networks, FFMNet gets higher performance with fewer parameters. Then FFMNet is split into two parts, which run on a mobile device and an edge server respectively. The feature generated by the mobile device is transmitted through the wireless channel to the edge server. To reduce the transmission overhead of the intermediate feature, a deep JSCC network is designed. By combining two networks together, the whole model achieves 512x compression for the intermediate feature and a performance loss within 2% on both tasks. At last, by training with noise, the FFMNet with JSCC is robust to various channel conditions and outperforms the separate source and channel coding scheme.