mlsanityData to 7 Oct 2026

Visual Intelligence through Human Interaction

arXiv 2111.06913 ↗First posted 12 Nov 2021cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Over the last decade, Computer Vision, the branch of Artificial Intelligence aimed at understanding the visual world, has evolved from simply recognizing objects in images to describing pictures, answering questions about images, aiding robots maneuver around physical spaces and even generating novel visual content. As these tasks and applications have modernized, so too has the reliance on more data, either for model training or for evaluation. In this chapter, we demonstrate that novel interaction strategies can enable new forms of data collection and evaluation for Computer Vision. First, we present a crowdsourcing interface for speeding up paid data collection by an order of magnitude, feeding the data-hungry nature of modern vision models. Second, we explore a method to increase volunteer contributions using automated social interventions. Third, we develop a system to ensure human evaluation of generative vision models are reliable, affordable and grounded in psychophysics theory. We conclude with future opportunities for Human-Computer Interaction to aid Computer Vision.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 65 of its 312 references that are in the map. First posted on arXiv 12 Nov 2021.

Builds on

65 of 312 references.

References · 65

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  9. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  10. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  11. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  12. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
38 more
  1. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  2. 2016Improved Techniques for Training GANscited by 2,938
  3. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  5. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  6. 2015VQA: Visual Question Answeringcited by 2,489
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2017Improved Training of Wasserstein GANscited by 2,223
  9. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  10. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  11. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  12. 2016Adversarial examples in the physical worldcited by 1,762
  13. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  14. 2018Demystifying MMD GANscited by 1,268
  15. 2018Model Cards for Model Reportingcited by 1,068
  16. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  17. 2018Assessing Generative Models via Precision and Recallcited by 401
  18. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 345
  19. 2016Visual Relationship Detection with Language Priorscited by 333
  20. 2017A Neural Representation of Sketch Drawingscited by 311
  21. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  22. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 262
  23. 2015YFCC100M: The New Data in Multimedia Researchcited by 234
  24. 2018A Note on the Inception Scorecited by 225
  25. 2018Pros and Cons of GAN Evaluation Measurescited by 224
  26. 2015Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networkscited by 214
  27. 2020Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systemscited by 138
  28. 2015Character-Aware Neural Language Modelscited by 129
  29. 2019Unifying Human and Statistical Evaluation for Natural Language Generationcited by 127
  30. 2017BEGAN: Boundary Equilibrium Generative Adversarial Networkscited by 121
  31. 2014The Rise of Social Botscited by 96
  32. 2019Photo-Sketching: Inferring Contour Drawings from Imagescited by 67
  33. 2020SketchyCOCO: Image Generation from Freehand Scene Sketchescited by 67
  34. 2019HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Modelscited by 59
  35. 2017Variational Approaches for Auto-Encoding Generative Adversarial Networkscited by 52
  36. 2020A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Taskscited by 52
  37. 2020Conceptual Metaphors Impact Perceptions of Human-AI Collaborationcited by 47
  38. 2019Information Maximizing Visual Question Generationcited by 40

50 most cited of 65 All 65 (JSON)

Built on by

Not cited yet.