# SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model

arXiv: https://arxiv.org/abs/2412.01550 · HTML: https://mlsanity.com/paper/2412.01550/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 16 of its 55 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 16 of 16)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Ferret: Refer and Ground Anything Anywhere at Any Granularity](https://mlsanity.com/paper/2310.07704/) (2023)
- [PointLLM: Empowering Large Language Models to Understand Point Clouds](https://mlsanity.com/paper/2308.16911/) (2023)
- [GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest](https://mlsanity.com/paper/2307.03601/) (2023)
- [ShapeLLM: Universal 3D Object Understanding for Embodied Interaction](https://mlsanity.com/paper/2402.17766/) (2024)
- [InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language](https://mlsanity.com/paper/2305.05662/) (2023)
- [RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulation](https://mlsanity.com/paper/2411.02704/) (2024)
- [PartAfford: Part-level Affordance Discovery from 3D Objects](https://mlsanity.com/paper/2202.13519/) (2022)
- [GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding](https://mlsanity.com/paper/2411.19626/) (2024)
- [Learning 2D Invariant Affordance Knowledge for 3D Affordance Grounding](https://mlsanity.com/paper/2408.13024/) (2024)
- [3D-TAFS: A Training-free Framework for 3D Affordance Segmentation](https://mlsanity.com/paper/2409.10078/) (2024)

## Built on by (most cited first, 18 of 18)

- [AffordDP: Generalizable Diffusion Policy with Transferable Affordance](https://mlsanity.com/paper/2412.03142/) (2024)
- [PartNeXt: A Next-Generation Dataset for Fine-Grained and Hierarchical 3D Part Understanding](https://mlsanity.com/paper/2510.20155/) (2025)
- [OpenHOI: Open-World Hand-Object Interaction Synthesis with Multimodal Large Language Model](https://mlsanity.com/paper/2505.18947/) (2025)
- [Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Context](https://mlsanity.com/paper/2510.20229/) (2025)
- [Don't Waste Bits! Adaptive KV-Cache Quantization for Lightweight On-Device LLMs](https://mlsanity.com/paper/2604.04722/) (2026)
- [Affogato: Open-Vocabulary Affordance Grounding with Automated Data Generation at Scale](https://mlsanity.com/paper/2506.12009/) (2025)
- [Diffusion Models are Open-World Affordance Learners: Leveraging Generative Priors for 3D Affordance Learning](https://mlsanity.com/paper/2508.01651/) (2025)
- [The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Models](https://mlsanity.com/paper/2601.08876/) (2026)
- [HAMMER: Harnessing MLLM via Cross-Modal Integration for Intention-Driven 3D Affordance Grounding](https://mlsanity.com/paper/2603.02329/) (2026)
- [CompassAD: Intent-Driven 3D Affordance Grounding in Functionally Competing Objects](https://mlsanity.com/paper/2604.02060/) (2026)
- [YOLOA: Real-Time Affordance Detection via LLM Adapter](https://mlsanity.com/paper/2512.03418/) (2025)
- [Part-Aware Open-Vocabulary 3D Affordance Grounding via Prototypical Semantic and Geometric Alignment](https://mlsanity.com/paper/2603.17647/) (2026)
- [AffordMatcher: Affordance Learning in 3D Scenes from Visual Signifiers](https://mlsanity.com/paper/2603.27970/) (2026)
- [SceneTeract: Probing and Improving Agent-Aware Activity Reasoning in 3D Indoor Scenes](https://mlsanity.com/paper/2603.29798/) (2026)
- [VoxAfford: Multi-Scale Voxel-Token Fusion for Open-Vocabulary 3D Affordance Detection](https://mlsanity.com/paper/2605.01365/) (2026)
- [LASAR: Towards Spatio-temporal Reasoning with Latent Cognitive Map](https://mlsanity.com/paper/2605.16899/) (2026)
- [Zero-shot 2D Grounding with Novel Affordance Types](https://mlsanity.com/paper/2608.08929/) (2026)
- [UniAfford: Token-Routed Multitask Learning for Generalizable 2D-3D Affordance Perception](https://mlsanity.com/paper/2609.37264/) (2026)

## Abstract (arXiv)

3D affordance segmentation aims to link human instructions to touchable regions of 3D objects for embodied manipulations. Existing efforts typically adhere to single-object, single-affordance paradigms, where each affordance type or explicit instruction strictly corresponds to a specific affordance region and are unable to handle long-horizon tasks. Such a paradigm cannot actively reason about complex user intentions that often imply sequential affordances. In this paper, we introduce the Sequential 3D Affordance Reasoning task, which extends the traditional paradigm by reasoning from cumbersome user intentions and then decomposing them into a series of segmentation maps. Toward this, we construct the first instruction-based affordance segmentation benchmark that includes reasoning over both single and sequential affordances, comprising 180K instruction-point cloud pairs. Based on the benchmark, we propose our model, SeqAfford, to unlock the 3D multi-modal large language model with additional affordance segmentation abilities, which ensures reasoning with world knowledge and fine-grained affordance grounding in a cohesive framework. We further introduce a multi-granular language-point integration module to endow 3D dense prediction. Extensive experimental evaluations show that our model excels over well-established methods and exhibits open-world generalization with sequential reasoning abilities.