# MARS: Multimodal Active Robotic Sensing for Articulated Characterization

arXiv: https://arxiv.org/abs/2407.01191 · HTML: https://mlsanity.com/paper/2407.01191/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 42 references that are in the map.
- First posted on arXiv 1 Jul 2024.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Habitat 2.0: Training Home Assistants to Rearrange their Habitat](https://mlsanity.com/paper/2106.14405/) (2021)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [Motion Representations for Articulated Animation](https://mlsanity.com/paper/2104.11280/) (2021)
- [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021)
- [EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection](https://mlsanity.com/paper/2007.08856/) (2020)
- [MVX-Net: Multimodal VoxelNet for 3D Object Detection](https://mlsanity.com/paper/1904.01649/) (2019)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [Ditto: Building Digital Twins of Articulated Objects from Interaction](https://mlsanity.com/paper/2202.08227/) (2022)
- [GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Parts](https://mlsanity.com/paper/2211.05272/) (2022)
- [Shape2Motion: Joint Analysis of Motion Parts and Attributes from 3D Shapes](https://mlsanity.com/paper/1903.03911/) (2019)
- [LASR: Learning Articulated Shape Reconstruction from a Monocular Video](https://mlsanity.com/paper/2105.02976/) (2021)
- [Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion](https://mlsanity.com/paper/2203.09780/) (2022)
- [VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects](https://mlsanity.com/paper/2106.14440/) (2021)
- [ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory](https://mlsanity.com/paper/2008.10518/) (2020)
- [Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction](https://mlsanity.com/paper/2012.01451/) (2020)
- [Deep Part Induction from Articulated Object Pairs](https://mlsanity.com/paper/1809.07417/) (2018)
- [AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions](https://mlsanity.com/paper/2112.00246/) (2021)
- [A Dataset for Developing and Benchmarking Active Vision](https://mlsanity.com/paper/1702.08272/) (2017)
- [4D-Net for Learned Multi-Modal Alignment](https://mlsanity.com/paper/2109.01066/) (2021)
- [Visual Identification of Articulated Object Parts](https://mlsanity.com/paper/2012.00284/) (2020)
- [Self-Supervised Learning of Part Mobility from Point Cloud Sequence](https://mlsanity.com/paper/2010.11735/) (2020)
- [Cross-Modality 3D Object Detection](https://mlsanity.com/paper/2008.10436/) (2020)

## Built on by (most cited first, 5 of 5)

- [Articulate AnyMesh: Open-Vocabulary 3D Articulated Objects Modeling](https://mlsanity.com/paper/2502.02590/) (2025)
- [FreeArt3D: Training-Free Articulated Object Generation using 3D Diffusion](https://mlsanity.com/paper/2510.25765/) (2025)
- [LARM: A Large Articulated-Object Reconstruction Model](https://mlsanity.com/paper/2511.11563/) (2025)
- [ArtPro: Self-Supervised Articulated Object Reconstruction with Adaptive Integration of Mobility Proposals](https://mlsanity.com/paper/2602.22666/) (2026)
- [Canonical Space Representation for 4D Panoptic Segmentation of Articulated Objects](https://mlsanity.com/paper/2511.05356/) (2025)

## Abstract (arXiv)

Precise perception of articulated objects is vital for empowering service robots. Recent studies mainly focus on point cloud, a single-modal approach, often neglecting vital texture and lighting details and assuming ideal conditions like optimal viewpoints, unrepresentative of real-world scenarios. To address these limitations, we introduce MARS, a novel framework for articulated object characterization. It features a multi-modal fusion module utilizing multi-scale RGB features to enhance point cloud features, coupled with reinforcement learning-based active sensing for autonomous optimization of observation viewpoints. In experiments conducted with various articulated object instances from the PartNet-Mobility dataset, our method outperformed current state-of-the-art methods in joint parameter estimation accuracy. Additionally, through active sensing, MARS further reduces errors, demonstrating enhanced efficiency in handling suboptimal viewpoints. Furthermore, our method effectively generalizes to real-world articulated objects, enhancing robot interactions. Code is available at https://github.com/robhlzeng/MARS.