mlsanityData to 14 Oct 2026

Y-MAP-Net: Real-time depth, normals, segmentation, multi-label captioning and 2D human pose in RGB images

arXiv 2411.10334 ↗First posted 15 Nov 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We present Y-MAP-Net, a Y-shaped neural network architecture designed for real-time multi-task learning on RGB images. Y-MAP-Net, simultaneously predicts depth, surface normals, human pose, semantic segmentation and generates multi-label captions, all from a single network evaluation. To achieve this, we adopt a multi-teacher, single-student training paradigm, where task-specific foundation models supervise the network's learning, enabling it to distill their capabilities into a lightweight architecture suitable for real-time applications. Y-MAP-Net, exhibits strong generalization, simplicity and computational efficiency, making it ideal for robotics and other practical scenarios. To support future research, we will release our code publicly.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 51 of its 65 references that are in the map. First posted on arXiv 15 Nov 2024.

Builds on

51 of 65 references.

References · 51

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  8. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  9. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  10. 2023Segment Anythingcited by 8,234
  11. 2017Mask R-CNNcited by 7,277
  12. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
38 more
  1. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  3. 2014Going Deeper with Convolutionscited by 4,241
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  8. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  9. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  10. 2024Depth Anything V2cited by 1,422
  11. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  12. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  13. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  14. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  15. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  16. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
  17. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  18. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  19. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 605
  20. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  21. 2015Empirical Evaluation of Rectified Activations in Convolutional Networkcited by 519
  22. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 382
  23. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
  24. 2013DeepPose: Human Pose Estimation via Deep Neural Networkscited by 305
  25. 2022Unifying Flow, Stereo and Depth Estimationcited by 275
  26. 2019HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimationcited by 247
  27. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  28. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 227
  29. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 221
  30. 2015DenseCap: Fully Convolutional Localization Networks for Dense Captioningcited by 176
  31. 2014Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimationcited by 129
  32. 2021Lite-HRNet: A Lightweight High-Resolution Networkcited by 91
  33. 2018The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approachescited by 75
  34. 2024Attention Heads of Large Language Models: A Surveycited by 67
  35. 2023UniHCP: A Unified Model for Human-Centric Perceptionscited by 59
  36. 2017Convolutional Image Captioningcited by 52
  37. 2022DPText-DETR: Towards Better Scene Text Detection with Dynamic Points in Transformercited by 50
  38. 2018Y-Net: Joint Segmentation and Classification for Diagnosis of Breast Biopsy Imagescited by 42

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.