# Continual Road-Scene Semantic Segmentation via Feature-Aligned Symmetric Multi-Modal Network

arXiv: https://arxiv.org/abs/2308.04702 · HTML: https://mlsanity.com/paper/2308.04702/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 9 Aug 2023.

## Builds on (most cited first, 16 of 16)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2203.08195/) (2022)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [PLOP: Learning without Forgetting for Continual Semantic Segmentation](https://mlsanity.com/paper/2011.11390/) (2020)
- [Incremental Learning Techniques for Semantic Segmentation](https://mlsanity.com/paper/1907.13372/) (2019)
- [xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/1911.12676/) (2019)
- [Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations](https://mlsanity.com/paper/2103.06342/) (2021)
- [Multi-view PointNet for 3D Scene Understanding](https://mlsanity.com/paper/1909.13603/) (2019)
- [Incremental Learning in Semantic Segmentation from Image Labels](https://mlsanity.com/paper/2112.01882/) (2021)
- [A Unified Point-Based Framework for 3D Segmentation](https://mlsanity.com/paper/1908.00478/) (2019)
- [FuseSeg: LiDAR Point Cloud Segmentation Fusing Multi-Modal Data](https://mlsanity.com/paper/1912.08487/) (2019)
- [Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels](https://mlsanity.com/paper/2005.06050/) (2020)
- [SELMA: SEmantic Large-scale Multimodal Acquisitions in Variable Weather, Daytime and Viewpoints](https://mlsanity.com/paper/2204.09788/) (2022)
- [Robust Semantic Segmentation in Adverse Weather Conditions by means of Sensor Data Fusion](https://mlsanity.com/paper/1905.10117/) (2019)
- [Continual Learning for LiDAR Semantic Segmentation: Class-Incremental and Coarse-to-Fine strategies on Sparse Data](https://mlsanity.com/paper/2304.03980/) (2023)
- [DepthFormer: Multimodal Positional Encodings and Cross-Input Attention for Transformer-Based Segmentation Networks](https://mlsanity.com/paper/2211.04188/) (2022)

## Built on by (most cited first, 6 of 6)

- [SDSTrack: Self-Distillation Symmetric Adapter Learning for Multi-Modal Visual Object Tracking](https://mlsanity.com/paper/2403.16002/) (2024)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [Modality-Incremental Learning with Disjoint Relevance Mapping Networks for Image-based Semantic Segmentation](https://mlsanity.com/paper/2411.17610/) (2024)
- [Cross-Architecture Auxiliary Feature Space Translation for Efficient Few-Shot Personalized Object Detection](https://mlsanity.com/paper/2407.01193/) (2024)
- [FedPromo: Federated Lightweight Proxy Models at the Edge Bring New Domains to Foundation Models](https://mlsanity.com/paper/2508.03356/) (2025)
- [MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment](https://mlsanity.com/paper/2509.14001/) (2025)

## Abstract (arXiv)

State-of-the-art multimodal semantic segmentation strategies combining LiDAR and color data are usually designed on top of asymmetric information-sharing schemes and assume that both modalities are always available. This strong assumption may not hold in real-world scenarios, where sensors are prone to failure or can face adverse conditions that make the acquired information unreliable. This problem is exacerbated when continual learning scenarios are considered since they have stringent data reliability constraints. In this work, we re-frame the task of multimodal semantic segmentation by enforcing a tightly coupled feature representation and a symmetric information-sharing scheme, which allows our approach to work even when one of the input modalities is missing. We also introduce an ad-hoc class-incremental continual learning scheme, proving our approach's effectiveness and reliability even in safety-critical settings, such as autonomous driving. We evaluate our approach on the SemanticKITTI dataset, achieving impressive performances.