mlsanityData to 7 Oct 2026

Tuning computer vision models with task rewards

arXiv 2302.08242 ↗First posted 16 Feb 2023cs.CVcited by 36

Not on the map. Why?

Abstract (arXiv)

Misalignment between model predictions and intended usage can be detrimental for the deployment of computer vision models. The issue is exacerbated when the task involves complex structured outputs, as it becomes harder to design procedures which address this misalignment. In natural language processing, this is often addressed using reinforcement learning techniques that align models with a task reward. We adopt this approach and show its surprising effectiveness across multiple computer vision tasks, such as object detection, panoptic segmentation, colorization and image captioning. We believe this approach has the potential to be widely useful for better aligning models with a diverse range of computer vision tasks.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 37 of its 48 references that are in the map. First posted on arXiv 16 Feb 2023.

Builds on

37 of 48 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  10. 2020Learning to summarize from human feedbackcited by 2,562
  11. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  12. 2021Scaling Vision Transformerscited by 993
25 more
  1. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  2. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  3. 2018Panoptic Segmentationcited by 796
  4. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  5. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  6. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  7. 2016Self-critical Sequence Training for Image Captioningcited by 544
  8. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  9. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 345
  10. 2019Generalization in Generation: A closer look at Exposure Biascited by 123
  11. 2015Minimum Risk Training for Neural Machine Translationcited by 114
  12. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  13. 2022UViM: A Unified Modeling Approach for Vision with Learned Guiding Codescited by 77
  14. 2018Can Neural Machine Translation be Improved with User Feedback?cited by 71
  15. 2021Machine Translation Decoding beyond Beam Searchcited by 55
  16. 2021Deep Reinforcement Learning in Computer Vision: A Comprehensive Surveycited by 42
  17. 2018Deep Reinforcement Learning For Sequence to Sequence Modelscited by 39
  18. 2019Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Taskcited by 39
  19. 2022End-to-End Transformer Based Model for Image Captioningcited by 37
  20. 2016End-to-end training of object class detectors for mean average precisioncited by 35
  21. 2015Training Deep Neural Networks via Direct Loss Minimizationcited by 25
  22. 2022Exploiting Multiple Sequence Lengths in Fast End to End Training for Image Captioningcited by 18
  23. 2016PoseAgent: Budget-Constrained 6D Object Pose Estimation via Reinforcement Learningcited by 9
  24. 2021MetricOpt: Learning to Optimize Black-Box Evaluation Metricscited by 9
  25. 2014Closed-Form Training of Conditional Random Fields for Large Scale Image Segmentationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 11; 2025: 5; 2026: 5.

36 papers.

Sort:

By area:

  1. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  2. 2023HIVE: Harnessing Human Feedback for Instructional Visual Editingcited by 172
  3. 20234M: Massively Multimodal Masked Modelingcited by 118
  4. 2023LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluationcited by 85
  5. 2024BOND: Aligning LLMs with Best-of-N Distillationcited by 73
  6. 2023Large AI Models in Health Informatics: Applications, Challenges, and the Futurecited by 61
  7. 2023Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Modelscited by 37
  8. 2024LocCa: Visual Pretraining with Location-aware Captionerscited by 35
  9. 2024ReinDiffuse: Crafting Physically Plausible Motions with Reinforced Diffusion Modelcited by 25
  10. 2023Carve3D: Improving Multi-view Reconstruction Consistency for Diffusion Models with RL Finetuningcited by 22
  11. 2023Reinforced UI Instruction Grounding: Towards a Generic UI Task Automation APIcited by 18
  12. 2023UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletonscited by 15
24 more
  1. 2023Cross-Domain Image Captioning with Discriminative Finetuningcited by 14
  2. 2023Reward Finetuning for Faster and More Accurate Unsupervised Object Discoverycited by 14
  3. 2025SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewardscited by 14
  4. 2023TaskMet: Task-Driven Metric Learning for Model Learningcited by 11
  5. 2025From Screens to Scenes: A Survey of Embodied AI in Healthcarecited by 10
  6. 2024Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformerscited by 8
  7. 2025Graph-Augmented Reasoning: Evolving Step-by-Step Knowledge Graph Retrieval for LLM Reasoningcited by 8
  8. 2023Uncertainty Estimation for Safety-critical Scene Segmentation via Fine-grained Reward Maximizationcited by 7
  9. 2025Large (Vision) Language Models are Unsupervised In-Context Learnerscited by 6
  10. 2023An Invitation to Deep Reinforcement Learningcited by 5
  11. 2024Lane2Seq: Towards Unified Lane Detection via Sequence Generationcited by 5
  12. 2024No Detail Left Behind: Revisiting Self-Retrieval for Fine-Grained Image Captioningcited by 5
  13. 2024RoboKoop: Efficient Control Conditioned Representations from Visual Input in Robotics using Koopman Operatorcited by 4
  14. 2024Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithmscited by 3
  15. 2025Test-Time Adaptation with Binary Feedbackcited by 3
  16. 2024Optimizing Novelty of Top-k Recommendations using Large Language Models and Reinforcement Learningcited by 2
  17. 2024Evolutionary Retrofittingcited by 2
  18. 2026EchoAvatar: Real-time Generative Avatar Animation from Audio Streamscited by 1
  19. 2023A dynamical clipping approach with task feedback for Proximal Policy Optimizationcited by 0
  20. 2024Incorporating System-level Safety Requirements in Perception Models via Reinforcement Learningcited by 0
  21. 2026Policy Gradient with Adaptive Entropy Annealing for Continual Fine-Tuningcited by 0
  22. 2026On the Feasibility and Opportunity of Autoregressive 3D Object Detectioncited by 0
  23. 2026Falcon Perception-HD: High Density Perception via Reinforcement Learningcited by 0
  24. 2026LookThere! Sparse Vision by Reinforced Selectioncited by 0