mlsanityData to 7 Oct 2026

MTLSegFormer: Multi-task Learning with Transformers for Semantic Segmentation in Precision Agriculture

arXiv 2305.02813 ↗First posted 4 May 2023cs.CV cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Multi-task learning has proven to be effective in improving the performance of correlated tasks. Most of the existing methods use a backbone to extract initial features with independent branches for each task, and the exchange of information between the branches usually occurs through the concatenation or sum of the feature maps of the branches. However, this type of information exchange does not directly consider the local characteristics of the image nor the level of importance or correlation between the tasks. In this paper, we propose a semantic segmentation method, MTLSegFormer, which combines multi-task learning and attention mechanisms. After the backbone feature extraction, two feature maps are learned for each task. The first map is proposed to learn features related to its task, while the second map is obtained by applying learned visual attention to locally re-weigh the feature maps of the other tasks. In this way, weights are assigned to local regions of the image of other tasks that have greater importance for the specific task. Finally, the two maps are combined and used to solve a task. We tested the performance in two challenging problems with correlated tasks and observed a significant improvement in accuracy, mainly in tasks with high dependence on the others.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 34 of its 47 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

34 of 47 references.

References · 34

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  7. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  8. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  10. 2021Vision Transformers for Dense Predictioncited by 1,789
  11. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  12. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
22 more
  1. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  2. 2017A Survey on Multi-Task Learningcited by 838
  3. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  4. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  5. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
  6. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  7. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  8. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  9. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 374
  10. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  11. 2021UniT: Multimodal Multitask Learning with a Unified Transformercited by 195
  12. 2019Attentive Single-Tasking of Multiple Taskscited by 148
  13. 2015An open access repository of images on plant health to enable the development of mobile disease diagnosticscited by 132
  14. 2017One Model To Learn Them Allcited by 86
  15. 2022MulT: An End-to-End Multitask Learning Transformercited by 47
  16. 2021ConvNets vs. Transformers: Whose Visual Representations are More Transferable?cited by 35
  17. 2020A Brief Review of Deep Multi-task Learning and Auxiliary Task Learningcited by 18
  18. 2020MT-BioNER: Multi-task Learning for Biomedical Named Entity Recognition using Deep Bidirectional Transformerscited by 14
  19. 2020A CNN Approach to Simultaneously Count Plants and Detect Plantation-Rows from UAV Imagerycited by 11
  20. 2017Analyzing Modular CNN Architectures for Joint Depth Prediction and Semantic Segmentationcited by 8
  21. 2020Multi-task deep learning for image segmentation using recursive approximation taskscited by 3
  22. 2021A Deep Learning Approach Based on Graphs to Detect Plantation Linescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025M3Net: Multimodal Multi-task Learning for 3D Detection, Segmentation, and Occupancy Prediction in Autonomous Drivingcited by 10
  2. 2025Improving underwater semantic segmentation with underwater image quality attention and muti-scale aggregation attentioncited by 4
  3. 2026WeedRepFormer: Reparameterizable Vision Transformers for Real-Time Waterhemp Segmentation and Gender Classificationcited by 0