mlsanityData to 14 Oct 2026

NepOOC-M: Bilingual Nepali-English Benchmark and Comparative Analysis of Multimodal Architectures for OOC Detection

arXiv 2608.19212 ↗First posted 18 Jun 2026cs.CL cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Out-of-context (OOC) misinformation pairs authentic images with misleading captions to construct false narratives without image manipulation, making detection a problem of multimodal alignment rather than image forensics. Despite the prevalence and consequences of OOC misinformation in Nepal, no public benchmark exists for Nepali. We introduce NepOOC, the first publicly available Nepali-dominant multilingual OOC benchmark, comprising 1,090 image-caption pairs (545 pristine, 545 OOC) annotated across five typologies (fabricated, miscaptioned, temporal mismatch, geographic mismatch, identity mismatch) with inter-annotator agreement kappa = 0.84. Systematic evaluation of five multimodal architectures alongside text-only and image-only baselines reveals that caption semantics appear sufficient for strong performance at the current dataset scale. A text-only mBERT model achieves 94.65+/-0.20% Macro-F1, statistically equivalent to the best multimodal system (ResNet-50+mBERT, 94.65+/-0.20%; McNemar median p = 1.000, 0/5 seeds significant at alpha = 0.05). Image-only models perform near chance (33-50%), while training-size scaling suggests that dataset expansion is a more direct path to progress than architectural sophistication or regional specialisation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 43 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

23 of 43 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  8. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  9. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  10. 2019Attention is not Explanationcited by 929
  11. 2021MuRIL: Multilingual Representations for Indian Languagescited by 179
  12. 2021NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Mediacited by 107
11 more
  1. 2024SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detectioncited by 74
  2. 2021Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resourcescited by 71
  3. 2022MuMiN: A Large-Scale Multilingual Multimodal Fact-Checked Misinformation Social Network Datasetcited by 61
  4. 2021Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multimodal Cluescited by 48
  5. 2023VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Biascited by 39
  6. 2024Multimodal Misinformation Detection using Large Vision-Language Modelscited by 21
  7. 2023RED-DOT: Multimodal Fact-checking via Relevant Evidence Detectioncited by 18
  8. 2025COVE: COntext and VEracity prediction for out-of-context imagescited by 14
  9. 2023Interpretable Detection of Out-of-Context Misinformation with Neural-Symbolic-Enhanced Large Multimodal Modelcited by 8
  10. 2025Evidence-Grounded Multimodal Misinformation Detection with Attention-Based GNNscited by 2
  11. 2023GRaMuFeN: Graph-based Multi-modal Fake News Detection in Social Mediacited by 1

Built on by

Not cited yet.