# HLTCOE Evaluation Team at TREC 2025: VQA Track

arXiv: https://arxiv.org/abs/2512.07738 · HTML: https://mlsanity.com/paper/2512.07738/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 12 references that are in the map.
- First posted on arXiv 8 Dec 2025.

## Builds on (most cited first, 8 of 8)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models](https://mlsanity.com/paper/2309.04461/) (2023)
- [Assessing Modality Bias in Video Question Answering Benchmarks with Multimodal Large Language Models](https://mlsanity.com/paper/2408.12763/) (2024)
- [Beyond VQA: Generating Multi-word Answer and Rationale to Visual Questions](https://mlsanity.com/paper/2010.12852/) (2020)
- [Generate then Select: Open-ended Visual Question Answering Guided by World Knowledge](https://mlsanity.com/paper/2305.18842/) (2023)

## Abstract (arXiv)

The HLTCOE Evaluation team participated in TREC VQA's Answer Generation (AG) task, for which we developed a listwise learning framework that aims to improve semantic precision and ranking consistency in answer generation. Given a video-question pair, a base multimodal model first generates multiple candidate answers, which are then reranked using a model trained with a novel Masked Pointer Cross-Entropy Loss with Rank Weights. This objective integrates pointer-based candidate selection, rank-dependent weighting, and masked cross-entropy under vocabulary restriction, enabling stable and interpretable listwise optimization. By bridging generative modeling with discriminative ranking, our method produces coherent, fine-grained answer lists. Experiments reveal consistent gains in accuracy and ranking stability, especially for questions requiring temporal reasoning and semantic disambiguation.