mlsanityData to 7 Oct 2026

Obj2Seq: Formatting Objects as Sequences with Class Prompt for Visual Tasks

arXiv 2209.13948 ↗First posted 28 Sep 2022cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Visual tasks vary a lot in their output formats and concerned contents, therefore it is hard to process them with an identical structure. One main obstacle lies in the high-dimensional outputs in object-level visual tasks. In this paper, we propose an object-centric vision framework, Obj2Seq. Obj2Seq takes objects as basic units, and regards most object-level visual tasks as sequence generation problems of objects. Therefore, these visual tasks can be decoupled into two steps. First recognize objects of given categories, and then generate a sequence for each of these objects. The definition of the output sequences varies for different tasks, and the model is supervised by matching these sequences with ground-truth targets. Obj2Seq is able to flexibly determine input categories to satisfy customized requirements, and be easily extended to different visual tasks. When experimenting on MS COCO, Obj2Seq achieves 45.7% AP on object detection, 89.0% AP on multi-label classification and 65.0% AP on human pose estimation. These results demonstrate its potential to be generally applied to different visual tasks. Code has been made available at: https://github.com/CASIA-IVA-Lab/Obj2Seq.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 57 of its 74 references that are in the map. First posted on arXiv 28 Sep 2022.

Builds on

57 of 74 references.

References · 57

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  10. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  11. 2020End-to-End Object Detection with Transformerscited by 7,384
  12. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
38 more
  1. 2017Mask R-CNNcited by 7,259
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  5. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  6. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  7. 2022Visual Prompt Tuningcited by 1,674
  8. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  9. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  10. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  11. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  12. 2019Objects as Pointscited by 1,172
  13. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  14. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  15. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  16. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  17. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  18. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  19. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  20. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  21. 2021Conditional DETR for Fast Training Convergencecited by 422
  22. 2020Asymmetric Loss For Multi-Label Classificationcited by 379
  23. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 345
  24. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  25. 2019HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimationcited by 247
  26. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  27. 2019Distribution-Aware Coordinate Representation for Human Pose Estimationcited by 165
  28. 2020Rethinking Transformer-based Set Prediction for Object Detectioncited by 156
  29. 2018PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Modelcited by 133
  30. 2021Query2Label: A Simple Transformer Way to Multi-Label Classificationcited by 126
  31. 2019PifPaf: Composite Fields for Human Pose Estimationcited by 120
  32. 2021MST: Masked Self-Supervised Transformer for Visual Representationcited by 114
  33. 2021Pose Recognition with Cascade Transformerscited by 99
  34. 2021Multi-Task Self-Training for Learning General Representationscited by 85
  35. 2021ML-Decoder: Scalable and Versatile Classification Headcited by 78
  36. 2022YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Losscited by 66
  37. 2020Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognitioncited by 50
  38. 2019DirectPose: Direct End-to-End Multi-Person Pose Estimationcited by 49

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 9; 2024: 2; 2025: 1.

12 papers.

  1. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  2. 2023Mitigating Hallucination in Visual Language Models with Visual Supervisioncited by 49
  3. 2023Ord2Seq: Regarding Ordinal Regression as Label Sequence Predictioncited by 18
  4. 2023RIO: A Benchmark for Reasoning Intention-Oriented Objects in Open Environmentscited by 15
  5. 2023Visual Exemplar Driven Task-Prompting for Unified Perception in Autonomous Drivingcited by 8
  6. 2024SafaRi:Adaptive Sequence Transformer for Weakly Supervised Referring Expression Segmentationcited by 6
  7. 2023Token Boosting for Robust Self-Supervised Visual Transformer Pre-trainingcited by 4
  8. 2024Multi-rater Prompting for Ambiguous Medical Image Segmentationcited by 3
  9. 2023Efficient Masked Autoencoders with Self-Consistencycited by 2
  10. 2025Improving Token-based Object Detection with Videocited by 1
  11. 2023FreConv: Frequency Branch-and-Integration Convolutional Networkscited by 0
  12. 2023SCoRD: Subject-Conditional Relation Detection with Text-Augmented Datacited by 0