mlsanityData to 7 Oct 2026

EXIF as Language: Learning Cross-Modal Associations Between Images and Camera Metadata

arXiv 2301.04647 ↗First posted 11 Jan 2023cs.CV cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

We learn a visual representation that captures information about the camera that recorded a given photo. To do this, we train a multimodal embedding between image patches and the EXIF metadata that cameras automatically insert into image files. Our model represents this metadata by simply converting it to text and then processing it with a transformer. The features that we learn significantly outperform other self-supervised and supervised features on downstream image forensics and calibration tasks. In particular, we successfully localize spliced image regions "zero shot" by clustering the visual embeddings for all of the patches within an image.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 35 of its 86 references that are in the map. First posted on arXiv 11 Jan 2023.

Builds on

35 of 86 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  10. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
23 more
  1. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  2. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  3. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  4. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  5. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  6. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  7. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  8. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
  9. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
  10. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  11. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  12. 2015YFCC100M: The New Data in Multimedia Researchcited by 234
  13. 2015Learning Visual Features from Large Weakly Supervised Datacited by 165
  14. 2018Learning Rich Features for Image Manipulation Detectioncited by 154
  15. 2018Fighting Fake News: Image Splice Detection via Learned Self-Consistencycited by 136
  16. 2020Learning Visual Representations with Caption Annotationscited by 123
  17. 2018Noiseprint: a CNN-based camera model fingerprintcited by 114
  18. 2016Learning Visual N-Grams from Web Datacited by 95
  19. 2017Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN)cited by 64
  20. 2019Detecting Photoshopped Faces by Scripting Photoshopcited by 55
  21. 2017A Perceptual Measure for Deep Single Image Camera Calibrationcited by 45
  22. 2019A Full-Image Full-Resolution End-to-End-Trainable CNN Framework for Image Forgery Detectioncited by 15
  23. 2019Copy-Move Forgery Classification via Unsupervised Domain Adaptationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 9; 2026: 1.

17 papers.

Sort:

By area:

  1. 2024Binding Touch to Everything: Learning Unified Multimodal Tactile Representationscited by 94
  2. 2023Pretrained Language Models as Visual Planners for Human Assistancecited by 28
  3. 2023Generating Visual Scenes from Touchcited by 26
  4. 2025RANGE: Retrieval Augmented Neural Fields for Multi-Resolution Geo-Embeddingscited by 18
  5. 2024Iterated Learning Improves Compositionality in Large Vision-Language Modelscited by 17
  6. 2025Forensic Self-Descriptions Are All You Need for Zero-Shot Detection, Open-Set Source Attribution, and Clustering of AI-generated Imagescited by 11
  7. 2024Scale-adaptive UAV Geo-localization via Height-aware Partition Learningcited by 4
  8. 2025Bi-Level Optimization for Self-Supervised AI-Generated Face Detectioncited by 4
  9. 2025Self-Supervised Learning for Detecting AI-Generated Faces as Anomaliescited by 3
  10. 2025FontGuard: A Robust Font Watermarking Approach Leveraging Deep Font Knowledgecited by 3
  11. 2025Processing and acquisition traces in visual encoders: What does CLIP know about your camera?cited by 3
  12. 2026RAWild: Sensor-Agnostic RAW Object Detection via Physics-Guided Curve and Grid Modelingcited by 3
5 more
  1. 2024CoAPT: Context Attribute words for Prompt Tuningcited by 1
  2. 2025Fine-grained Defocus Blur Control for Generative Image Modelscited by 1
  3. 2024PhotoHolmes: a Python library for forgery detection in digital imagescited by 0
  4. 2025SNAP: A Benchmark for Testing the Effects of Capture Conditions on Fundamental Vision Taskscited by 0
  5. 2025Beyond Memorization: A Multi-Modal Ordinal Regression Benchmark to Expose Popularity Bias in Vision-Language Modelscited by 0