mlsanityData to 14 Oct 2026

QCalEval: Benchmarking Vision-Language Models for Quantum Calibration Plot Understanding

arXiv 2604.25884 ↗First posted 28 Apr 2026quant-ph cs.CVcited by 11

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Quantum computing calibration depends on interpreting experimental data, and calibration plots provide the most universal human-readable representation for this task, yet no systematic evaluation exists of how well vision-language models (VLMs) interpret them. We introduce QCalEval, the first VLM benchmark for quantum calibration plots: 243 samples across 87 scenario types from 22 experiment families, spanning superconducting qubits and neutral atoms, evaluated on six question types in both zero-shot and in-context learning settings. The best general-purpose zero-shot model reaches a mean score of 72.3, and many open-weight models degrade under multi-image in-context learning, whereas frontier closed models improve substantially. A supervised fine-tuning ablation at the 9-billion-parameter scale shows that SFT improves zero-shot performance but cannot close the multimodal in-context learning gap. As a reference case study, we release NVIDIA Ising Calibration 1, an open-weight model based on Qwen3.5-35B-A3B that reaches 74.7 zero-shot average score.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 56 references that are in the map. First posted on arXiv 28 Apr 2026.

Builds on

32 of 56 references.

References · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  7. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  8. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  9. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  10. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  11. 2025Kimi-VL Technical Reportcited by 388
  12. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 340
20 more
  1. 2019PlotQA: Reasoning over Scientific Plotscited by 321
  2. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
  3. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 163
  4. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 160
  5. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 160
  6. 2024Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Modelscited by 139
  7. 2024ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wildcited by 79
  8. 2021SciCap: Generating Captions for Scientific Figurescited by 77
  9. 2024ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoningcited by 68
  10. 2024Many-Shot In-Context Learning in Multimodal Foundation Modelscited by 67
  11. 2024ORGANA: A Robotic Assistant for Automated Chemistry Experimentation and Characterizationcited by 61
  12. 2024SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretationcited by 48
  13. 2023Errors are Useful Prompts: Instruction Guided Task Programming with Verifier-Assisted Iterative Promptingcited by 46
  14. 2024VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learningcited by 36
  15. 2024Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMscited by 30
  16. 2022Chart Question Answering: State of the Art and Future Directionscited by 26
  17. 2024Towards Multimodal In-Context Learning for Vision & Language Modelscited by 25
  18. 2024Agents for self-driving laboratories applied to quantum computingcited by 19
  19. 2026Toward Full Autonomous Laboratory Instrumentation Control with Large Language Modelscited by 5
  20. 2026Large Language Model-Assisted Superconducting Qubit Experimentscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026Qiskit QuantumKatas: Adapting Microsoft's Quantum Computing exercises for LLM evaluationcited by 2
  2. 2026Agentic AI for Scientific Reasoning in Autonomous Quantum Sensing Experimentscited by 2
  3. 2026RIZZ: Routing Interactions to Near Zero-Interference Zones for Continual Adaptation of Black-Box Agentscited by 1
  4. 2026Frozen Models, Evolving Expertise: Model-Agnostic Learning from Deployment Experience for Multimodal Medical AIcited by 1
  5. 2026RiverONE: Generating Knowledge-Intensive VLM by Simulated Quantum Machinescited by 0
  6. 2026When AI meets quantum information: A comprehensive reviewcited by 0
  7. 2026Self-Specializing Vision-Language Transmon Chip Calibration in a Physics-Grounded Environmentcited by 0
  8. 2026Action-Factored Multi-Agent Reinforcement Learning for Scalable Quantum Device Tuningcited by 0
  9. 2026Quantum Circuit Vision: Cost-Aware Evaluation of Visual AI Agents for Quantum Code Generationcited by 0
  10. 2026Machine Learning for Charge State Characterization of Isolated Double Quantum Dotscited by 0
  11. 2026Evaluating Verified Autonomy in Quantum Engineeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Fine-Tuning Large Language Models for Quantum Reasoning
  2. 2025CalibQuant: 1-Bit KV Cache Quantization for Multimodal LLMs
  3. 2026Rethinking Practical and Efficient Quantization Calibration for Vision-Language Models
  4. 2026Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery
  5. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
  6. 2023INT-FP-QSim: Mixed Precision and Formats For Large Language Models and Vision Transformers
  7. 2025Large Language Model Scaling Laws for Neural Quantum States in Quantum Chemistry
  8. 2025Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization