# SpectralWaste Dataset: Multimodal Data for Waste Sorting Automation

arXiv: https://arxiv.org/abs/2403.18033 · HTML: https://mlsanity.com/paper/2403.18033/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Builds on (most cited first, 15 of 15)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [COTR: Correspondence Transformer for Matching Across Images](https://mlsanity.com/paper/2103.14167/) (2021)
- [ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes](https://mlsanity.com/paper/2106.02740/) (2021)
- [VL-Grasp: a 6-Dof Interactive Grasp Policy for Language-Oriented Objects in Cluttered Indoor Scenes](https://mlsanity.com/paper/2308.00640/) (2023)
- [TACO: Trash Annotations in Context for Litter Detection](https://mlsanity.com/paper/2003.06975/) (2020)
- [Validating Hyperspectral Image Segmentation](https://mlsanity.com/paper/1811.03707/) (2018)
- [TRansPose: Large-Scale Multispectral Dataset for Transparent Object](https://mlsanity.com/paper/2307.05016/) (2023)
- [A Hyperspectral and RGB Dataset for Building Facade Segmentation](https://mlsanity.com/paper/2212.02749/) (2022)
- [VisDA 2022 Challenge: Domain Adaptation for Industrial Waste Sorting](https://mlsanity.com/paper/2303.14828/) (2023)
- [Cross-Modal Analysis of Human Detection for Robotics: An Industrial Case Study](https://mlsanity.com/paper/2108.01495/) (2021)
- [Non-myopic Planetary Exploration Combining In Situ and Remote Measurements](https://mlsanity.com/paper/1904.12255/) (2019)
- [Geomorphological Analysis Using Unpiloted Aircraft Systems, Structure from Motion, and Deep Learning](https://mlsanity.com/paper/1909.12874/) (2019)
- [SLURP! Spectroscopy of Liquids Using Robot Pre-Touch Sensing](https://mlsanity.com/paper/2210.04941/) (2022)

## Built on by (most cited first, 7 of 7)

- [COSNet: A Novel Semantic Segmentation Network using Enhanced Boundaries in Cluttered Scenes](https://mlsanity.com/paper/2410.24139/) (2024)
- [Electrolyzers-HSI: Close-Range Multi-Scene Hyperspectral Imaging Benchmark Dataset](https://mlsanity.com/paper/2505.20507/) (2025)
- [FusionSort: Enhanced Cluttered Waste Segmentation with Advanced Decoding and Comprehensive Modality Optimization](https://mlsanity.com/paper/2508.19798/) (2025)
- [Bidirectional Cross-Attention Fusion of High-Resolution RGB and Low-Resolution Hyperspectral Inputs for Multimodal Semantic Segmentation](https://mlsanity.com/paper/2603.13941/) (2026)
- [Robust and Label-Efficient Deep Waste Detection](https://mlsanity.com/paper/2508.18799/) (2025)
- [Towards Effective Waste Segmentation for Automated Waste Recycling in Cluttered Background](https://mlsanity.com/paper/2606.13587/) (2026)
- [SPECTRA-VLA: Base-Fine Hyperspectral Token Transfer at Frozen Vision-Language-Action Interfaces](https://mlsanity.com/paper/2610.19651/) (2026)

## Abstract (arXiv)

The increase in non-biodegradable waste is a worldwide concern. Recycling facilities play a crucial role, but their automation is hindered by the complex characteristics of waste recycling lines like clutter or object deformation. In addition, the lack of publicly available labeled data for these environments makes developing robust perception systems challenging. Our work explores the benefits of multimodal perception for object segmentation in real waste management scenarios. First, we present SpectralWaste, the first dataset collected from an operational plastic waste sorting facility that provides synchronized hyperspectral and conventional RGB images. This dataset contains labels for several categories of objects that commonly appear in sorting plants and need to be detected and separated from the main trash flow for several reasons, such as security in the management line or reuse. Additionally, we propose a pipeline employing different object segmentation architectures and evaluate the alternatives on our dataset, conducting an extensive analysis for both multimodal and unimodal alternatives. Our evaluation pays special attention to efficiency and suitability for real-time processing and demonstrates how HSI can bring a boost to RGB-only perception in these realistic industrial settings without much computational overhead.