# Asynchronous Events-based Panoptic Segmentation using Graph Mixer Neural Network

arXiv: https://arxiv.org/abs/2305.03640 · HTML: https://mlsanity.com/paper/2305.03640/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 5 May 2023.

## Builds on (most cited first, 20 of 20)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [MetaFormer Is Actually What You Need for Vision](https://mlsanity.com/paper/2111.11418/) (2021)
- [Graph U-Nets](https://mlsanity.com/paper/1905.05178/) (2019)
- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [AEGNN: Asynchronous Event-based Graph Neural Networks](https://mlsanity.com/paper/2203.17149/) (2022)
- [EV-SegNet: Semantic Segmentation for Event-based Cameras](https://mlsanity.com/paper/1811.12039/) (2018)
- [ESS: Learning Event-based Semantic Segmentation from Still Images](https://mlsanity.com/paper/2203.10016/) (2022)
- [PointMixer: MLP-Mixer for Point Cloud Understanding](https://mlsanity.com/paper/2111.11187/) (2021)
- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective](https://mlsanity.com/paper/2209.13232/) (2022)
- [Event Transformer. A sparse-aware solution for efficient event data processing](https://mlsanity.com/paper/2204.03355/) (2022)
- [Amodal Panoptic Segmentation](https://mlsanity.com/paper/2202.11542/) (2022)
- [TactileSGNet: A Spiking Graph Neural Network for Event-based Tactile Object Recognition](https://mlsanity.com/paper/2008.08046/) (2020)
- [Neuromorphic Camera Denoising using Graph Neural Network-driven Transformers](https://mlsanity.com/paper/2112.09685/) (2021)
- [A Neuromorphic Dataset for Object Segmentation in Indoor Cluttered Environment](https://mlsanity.com/paper/2302.06301/) (2023)
- [Bimodal SegNet: Instance Segmentation Fusing Events and RGB Frames for Robotic Grasping](https://mlsanity.com/paper/2303.11228/) (2023)
- [CM-MLP: Cascade Multi-scale MLP with Axial Context Relation Encoder for Edge Segmentation of Medical Image](https://mlsanity.com/paper/2208.10701/) (2022)

## Built on by (most cited first, 3 of 3)

- [Neuromorphic Vision-based Motion Segmentation with Graph Transformer Neural Network](https://mlsanity.com/paper/2404.10940/) (2024)
- [Efficient RGB-D Scene Understanding via Multi-task Adaptive Learning and Cross-dimensional Feature Guidance](https://mlsanity.com/paper/2603.07570/) (2026)
- [Multi-FRuGaL: Multimodal Flexible Redundancy-aware Decomposed Gated Learning for Cancer Diagnosis and Prognosis](https://mlsanity.com/paper/2606.06867/) (2026)

## Abstract (arXiv)

In the context of robotic grasping, object segmentation encounters several difficulties when faced with dynamic conditions such as real-time operation, occlusion, low lighting, motion blur, and object size variability. In response to these challenges, we propose the Graph Mixer Neural Network that includes a novel collaborative contextual mixing layer, applied to 3D event graphs formed on asynchronous events. The proposed layer is designed to spread spatiotemporal correlation within an event graph at four nearest neighbor levels parallelly. We evaluate the effectiveness of our proposed method on the Event-based Segmentation (ESD) Dataset, which includes five unique image degradation challenges, including occlusion, blur, brightness, trajectory, scale variance, and segmentation of known and unknown objects. The results show that our proposed approach outperforms state-of-the-art methods in terms of mean intersection over the union and pixel accuracy. Code available at: https://github.com/sanket0707/GNN-Mixer.git