# OpenVNA: A Framework for Analyzing the Behavior of Multimodal Language Understanding System under Noisy Scenarios

arXiv: https://arxiv.org/abs/2407.02773 · HTML: https://mlsanity.com/paper/2407.02773/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2024.

## Built on by (most cited first, 2 of 2)

- [UniSLU: Unified Spoken Language Understanding from Heterogeneous Cross-Task Datasets](https://mlsanity.com/paper/2507.12951/) (2025)
- [MultiMem: Measuring and Mitigating Memorization in Multi-Modal Contrastive Learning](https://mlsanity.com/paper/2606.22220/) (2026)

## Abstract (arXiv)

We present OpenVNA, an open-source framework designed for analyzing the behavior of multimodal language understanding systems under noisy conditions. OpenVNA serves as an intuitive toolkit tailored for researchers, facilitating convenience batch-level robustness evaluation and on-the-fly instance-level demonstration. It primarily features a benchmark Python library for assessing global model robustness, offering high flexibility and extensibility, thereby enabling customization with user-defined noise types and models. Additionally, a GUI-based interface has been developed to intuitively analyze local model behavior. In this paper, we delineate the design principles and utilization of the created library and GUI-based web platform. Currently, OpenVNA is publicly accessible at urlhttps://github.com/thuiar/OpenVNA, with a demonstration video available at urlhttps://youtu.be/0Z9cW7RGct4.