# 3DTMDet: A Dual-Path Synergy Network of Transformer and SSM for 3D Object Detection in Point Clouds

arXiv: https://arxiv.org/abs/2605.15546 · HTML: https://mlsanity.com/paper/2605.15546/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Point clouds, detection and remote sensing, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 15 May 2026.

## Placed in

- State-space sequence models (S4, Mamba) / [Point clouds, detection and remote sensing](https://mlsanity.com/ssm/point-clouds-detection-and-remote-sensing/)

## Builds on (most cited first, 15 of 15)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [Group-Free 3D Object Detection via Transformers](https://mlsanity.com/paper/2104.00678/) (2021)
- [DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets](https://mlsanity.com/paper/2301.06051/) (2023)
- [M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers](https://mlsanity.com/paper/2104.11896/) (2021)
- [Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detection](https://mlsanity.com/paper/2406.10700/) (2024)
- [LION: Linear Group RNN for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/2407.18232/) (2024)
- [PoinTramba: A Hybrid Transformer-Mamba Framework for Point Cloud Analysis](https://mlsanity.com/paper/2405.15463/) (2024)
- [MambaOcc: Visual State Space Model for BEV-based Occupancy Prediction with Local Adaptive Reordering](https://mlsanity.com/paper/2408.11464/) (2024)
- [A Survey of Deep Learning Based Radar and Vision Fusion for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/2406.00714/) (2024)

## Abstract (arXiv)

A fundamental challenge in point cloud object detection lies in the conflict between the extreme sparsity of distant points and the need for remote context understanding. The existing methods typically use 1D serialization to expand the receptive field, which inevitably discards already scarce local geometric details and reduces detection of distant and small objects. To address this issue, we propose 3DTMDet, a novel detection network that synergistically combines state space models (Mamba) with Transformers. The core idea is to utilize SSM's linear complexity and advantages in long sequence modeling to effectively capture global interactions between sparse and distant points, while using Transformer modules with local attention to encode fine-grained geometric structures in local point sets, preserving accurate shape information. We propose the 3D Hybrid Mamba Transformer (3DHMT) block, which uses an SSM-Attention-SSM pipeline to balance global context understanding and local detail preservation, effectively alleviating the tension between receptive field enlargement and geometric preservation in remote detection. In addition, we introduced a voxel generation block inspired by LiDAR physics, which diffuses features along the sensor observation direction to reconstruct the complete object structure of occlusion and distant areas. Extensive experiments conducted on the KITTI and ONCE datasets have shown that 3DTMDet outperforms state-of-the-art detectors. The code is available at https://github.com/QiuBingwen/3DTMDet.