mlsanityData to 7 Oct 2026

Cerberus Transformer: Joint Semantic, Affordance and Attribute Parsing

arXiv 2111.12608 ↗First posted 24 Nov 2021cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

Multi-task indoor scene understanding is widely considered as an intriguing formulation, as the affinity of different tasks may lead to improved performance. In this paper, we tackle the new problem of joint semantic, affordance and attribute parsing. However, successfully resolving it requires a model to capture long-range dependency, learn from weakly aligned data and properly balance sub-tasks during training. To this end, we propose an attention-based architecture named Cerberus and a tailored training framework. Our method effectively addresses the aforementioned challenges and achieves state-of-the-art performance on all three tasks. Moreover, an in-depth analysis shows concept affinity consistent with human cognition, which inspires us to explore the possibility of weakly supervised learning. Surprisingly, Cerberus achieves strong results using only 0.1%-1% annotation. Visualizations further confirm that this success is credited to common attention maps across tasks. Code and models can be accessed at https://github.com/OPEN-AIR-SUN/Cerberus.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 28 of its 47 references that are in the map. First posted on arXiv 24 Nov 2021.

Builds on

28 of 47 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2016Pyramid Scene Parsing Networkcited by 2,472
  4. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  5. 2021Vision Transformers for Dense Predictioncited by 1,789
  6. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  7. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  8. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  9. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  10. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  11. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 463
  12. 2021Group-Free 3D Object Detection via Transformerscited by 215
16 more
  1. 2020OccuSeg: Occupancy-aware 3D Instance Segmentationcited by 177
  2. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  3. 2018OCNet: Object Context Network for Scene Parsingcited by 140
  4. 2017AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detectioncited by 126
  5. 20203D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Priorcited by 124
  6. 2020Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentationcited by 120
  7. 2019ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentationcited by 105
  8. 2020Robust Learning Through Cross-Task Consistencycited by 105
  9. 2021Holistic 3D Scene Understanding from a Single Image with Implicit Representationcited by 94
  10. 2019Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsensecited by 71
  11. 2019FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentationcited by 70
  12. 2019Efficient Semantic Scene Completion Network with Spatial Group Convolutioncited by 66
  13. 2019Cascaded Context Pyramid for Full-Resolution 3D Semantic Scene Completioncited by 43
  14. 2017Physics Inspired Optimization on Semantic Transfer Features: An Alternative Method for Room Layout Estimationcited by 29
  15. 2021PQ-Transformer: Jointly Parsing 3D Objects and Layouts from Point Cloudscited by 26
  16. 2019What Object Should I Use? - Task Driven Object Detectioncited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 9; 2024: 5; 2025: 5; 2026: 2.

22 papers.

Sort:

By area:

  1. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  2. 2023Open-Vocabulary Affordance Detection in 3D Point Cloudscited by 41
  3. 2023LODE: Locally Conditioned Eikonal Implicit Scene Completion from Sparse LiDARcited by 31
  4. 2022TOIST: Task Oriented Instance Segmentation Transformer with Noun-Pronoun Distillationcited by 25
  5. 2023Delving into Shape-aware Zero-shot Semantic Segmentationcited by 16
  6. 2023Zero-1-to-3: Domain-level Zero-shot Cognitive Diagnosis via One Batch of Early-bird Students towards Three Diagnostic Objectivescited by 15
  7. 2025PartRM: Modeling Part-Level Dynamics with Large Cross-State Reconstruction Modelcited by 15
  8. 2024Adaptive Surface Normal Constraint for Geometric Estimation from Monocular Imagescited by 14
  9. 2024PreAfford: Universal Affordance-Based Pre-Grasping for Diverse Objects and Environmentscited by 11
  10. 2023STRAP: Structured Object Affordance Segmentation with Point Supervisioncited by 10
  11. 2025Weakly-Supervised Affordance Grounding Guided by Part-Level Semantic Priorscited by 9
  12. 2025Reusing Attention for One-stage Lane Topology Understandingcited by 9
10 more
  1. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 8
  2. 2023Sim-Suction: Learning a Suction Grasp Policy for Cluttered Environments Using a Synthetic Benchmarkcited by 7
  3. 2024Diffusion-based Visual Anagram as Multi-task Learningcited by 6
  4. 2023DPF: Learning Dense Prediction Fields with Weak Supervisioncited by 3
  5. 2025Interpretable Affordance Detection on 3D Point Clouds with Probabilistic Prototypescited by 3
  6. 2026PAM: A Pose-Appearance-Motion Engine for Sim-to-Real HOI Video Generationcited by 3
  7. 2024Locate n' Rotate: Two-stage Openable Part Detection with Foundation Model Priorscited by 1
  8. 2026Attention-Spectrum Regularization for Replay-Free Continual Multimodal LLMscited by 1
  9. 2023ECT: Fine-grained Edge Detection with Learned Cause Tokenscited by 0
  10. 2025Capturing Fine-Grained Alignments Improves 3D Affordance Detectioncited by 0