mlsanityData to 14 Oct 2026

IO Transformer: Evaluating SwinV2-Based Reward Models for Computer Vision

arXiv 2411.00252 ↗First posted 31 Oct 2024cs.CV cs.LGcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Transformers and their derivatives have achieved state-of-the-art performance across text, vision, and speech recognition tasks. However, minimal effort has been made to train transformers capable of evaluating the output quality of other models. This paper examines SwinV2-based reward models, called the Input-Output Transformer (IO Transformer) and the Output Transformer. These reward models can be leveraged for tasks such as inference quality evaluation, data categorization, and policy optimization. Our experiments demonstrate highly accurate model output quality assessment across domains where the output is entirely dependent on the input, with the IO Transformer achieving perfect evaluation accuracy on the Change Dataset 25 (CD25). We also explore modified Swin V2 architectures. Ultimately Swin V2 remains on top with a score of 95.41 % on the IO Segmentation Dataset, outperforming the IO Transformer in scenarios where the output is not entirely dependent on the input. Our work expands the application of transformer architectures to reward modeling in computer vision and provides critical insights into optimizing these models for various tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 27 of its 34 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

27 of 34 references.

References · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  10. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  11. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  12. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
15 more
  1. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
  2. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  4. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  6. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  7. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 830
  8. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  9. 2022A Transformer-Based Siamese Network for Change Detectioncited by 244
  10. 2015Learning to Compare Image Patches via Convolutional Neural Networkscited by 109
  11. 2021Learning and Planning in Complex Action Spacescited by 77
  12. 2022Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restorationcited by 73
  13. 2019TransMatch: A Transfer-Learning Scheme for Semi-Supervised Few-Shot Learningcited by 39
  14. 2017Deep Reinforcement Learning for Visual Object Tracking in Videoscited by 8
  15. 2022Deep Reinforcement Learning with Swin Transformerscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model
  2. 2026Understanding vision transformer robustness through the lens of out-of-distribution detection
  3. 2024Faster Inference of Integer SWIN Transformer by Removing the GELU Activation
  4. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey
  5. 2026DiT-Reward: Generative Representations for Text-to-Image Reward Modeling
  6. 2024ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers
  7. 2025Image Recognition with Online Lightweight Vision Transformer: A Survey
  8. 2025ViLBench: A Suite for Vision-Language Process Reward Modeling