mlsanityData to 14 Oct 2026

SegmATRon: Embodied Adaptive Semantic Segmentation for Indoor Environment

arXiv 2310.12031 ↗First posted 18 Oct 2023cs.CV cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper presents an adaptive transformer model named SegmATRon for embodied image semantic segmentation. Its distinctive feature is the adaptation of model weights during inference on several images using a hybrid multicomponent loss function. We studied this model on datasets collected in the photorealistic Habitat and the synthetic AI2-THOR Simulators. We showed that obtaining additional images using the agent's actions in an indoor environment can improve the quality of semantic segmentation. The code of the proposed approach and datasets are publicly available at https://github.com/wingrune/SegmATRon.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 57 references that are in the map. First posted on arXiv 18 Oct 2023.

Builds on

32 of 57 references.

References · 32

  1. 2023Segment Anythingcited by 8,234
  2. 2020End-to-End Object Detection with Transformerscited by 7,414
  3. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  4. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  5. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  7. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  8. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  9. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  10. 2019Panoptic-DeepLabcited by 363
  11. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 328
  12. 2020Hierarchical Multi-Scale Attention for Semantic Segmentationcited by 183
20 more
  1. 2020Video Panoptic Segmentationcited by 142
  2. 2022Habitat-Matterport 3D Semantics Datasetcited by 110
  3. 2021Visual Room Rearrangementcited by 102
  4. 2023DVIS: Decoupled Video Instance Segmentation Frameworkcited by 77
  5. 2021SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistencycited by 48
  6. 2022Retrospectives on the Embodied AI Workshopcited by 46
  7. 2021Polarized Self-Attention: Towards High-quality Pixel-wise Regressioncited by 39
  8. 2022Is Mapping Necessary for Realistic PointGoal Navigation?cited by 35
  9. 2023TarViS: A Unified Approach for Target-based Video Segmentationcited by 33
  10. 2023Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentationcited by 24
  11. 2022Interactron: Embodied Adaptive Object Detectioncited by 21
  12. 2023Mask-Free Video Instance Segmentationcited by 20
  13. 2020Embodied Visual Active Learning for Semantic Segmentationcited by 19
  14. 2022Ask4Help: Learning to Leverage an Expert for Embodied Taskscited by 18
  15. 2022Embodied Active Domain Adaptation for Semantic Segmentation via Informative Path Planningcited by 11
  16. 2023Learning to View: Decision Transformers for Active Object Detectioncited by 11
  17. 2022GE-Grasp: Efficient Target-Oriented Grasping in Dense Cluttercited by 9
  18. 2020Move to See Better: Self-Improving Embodied Object Detectioncited by 7
  19. 2023Video Object Segmentation in Panoptic Wild Scenescited by 6
  20. 2022Smart Explorer: Recognizing Objects in Dense Clutter via Interactive Explorationcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Interactive Semantic Map Representation for Skill-based Visual Object Navigationcited by 2