mlsanityData to 7 Oct 2026

Can Machines Imitate Humans? Integrative Turing-like tests for Language and Vision Demonstrate a Narrowing Gap

arXiv 2211.13087 ↗First posted 23 Nov 2022cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

As AI becomes increasingly embedded in daily life, ascertaining whether an agent is human is critical. We systematically benchmark AI's ability to imitate humans in three language tasks (image captioning, word association, conversation) and three vision tasks (color estimation, object detection, attention prediction), collecting data from 636 humans and 37 AI agents. Next, we conducted 72,191 Turing-like tests with 1,916 human judges and 10 AI judges. Current AIs are approaching the ability to convincingly impersonate humans and deceive human judges in both language and vision. Even simple AI judges outperformed humans in distinguishing AI from human responses. Imitation ability showed minimal correlation with conventional AI performance metrics, suggesting that passing as human is an important independent evaluation criterion. The large-scale Turing datasets and metrics introduced here offer valuable benchmarks for assessing human-likeness in AI and highlight the importance of rigorous, quantitative imitation tests for AI development.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 112 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

41 of 112 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2020End-to-End Object Detection with Transformerscited by 7,384
  10. 2017Mask R-CNNcited by 7,259
  11. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
29 more
  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  2. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  3. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  4. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  5. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  6. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  7. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  8. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  9. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  10. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  11. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  12. 2019Defending Against Neural Fake Newscited by 553
  13. 2016Self-critical Sequence Training for Image Captioningcited by 544
  14. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  15. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
  16. 2018nocaps: novel object captioning at scalecited by 444
  17. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  18. 2022Detecting Deepfakes with Self-Blended Imagescited by 332
  19. 2022A Survey on Generative Diffusion Modelcited by 238
  20. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  21. 2021Partial success in closing the gap between human and machine visioncited by 185
  22. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 96
  23. 2022Harmonizing the object recognition strategies of deep neural networks with humanscited by 59
  24. 2018Learning to Evaluate Image Captioningcited by 53
  25. 2021Transparent Human Evaluation for Image Captioningcited by 40
  26. 2022Large Language Models and the Reverse Turing Testcited by 38
  27. 2019Putting visual object recognition in contextcited by 33
  28. 2018Finding any Waldo: zero-shot invariant and efficient visual searchcited by 28
  29. 2021Visual Search Asymmetry: Deep Nets and Humans Share Similar Inherent Biasescited by 12

Built on by

Not cited yet.