EG-VQA: Benchmarking Verifiable Video Question
Answering with Grounded Temporal Evidence

Linpeng Huang1* Weixing Chen1* Zexin Chen1 Yang Liu1,3† Liang Lin1,2,3
1Sun Yat-sen University 2Peng Cheng Laboratory 3X-Era AI Lab
*Equal Contribution  †Corresponding Author
EG-VQA overview

Overview of evidence-grounded VideoQA evaluation in EG-VQA. (a) Conventional VideoQA evaluation primarily focuses on answer prediction, without explicitly verifying whether the prediction is supported by relevant video evidence. (b) EG-VQA requires models to generate answers together with temporally localized textual evidence, enabling joint evaluation of answer correctness and evidence faithfulness.

Abstract

Recent advances in Video Large Language Models (Video-LLMs) have yielded promising performance on Video Question Answering (VideoQA). Nevertheless, existing benchmarks are predominantly evaluated through answer correctness, while the faithfulness of predicted evidence supporting those answers remains insufficiently evaluated. This disconnect between answer generation and evidence verification motivates the construction of the Evidence-Grounded Video Question Answering Benchmark (EG-VQA), a large-scale open-ended benchmark in which each QA pair is annotated with temporally localized textual evidence, requiring models to jointly produce answers and verifiable evidence. EG-VQA comprises 2,067 videos and 11,838 QA pairs with fine-grained evidence annotations. To evaluate predicted evidence, Evidence-Grounded F1 (EG-F1) is introduced as a metric that jointly measures temporal alignment and semantic consistency between predicted and ground-truth evidence. Experiments reveal a substantial discrepancy between answer correctness and evidence faithfulness: even strong proprietary models can achieve high answer accuracy while exhibiting gaps in temporal-semantic grounding. To investigate evidence-aware learning under EG-VQA, we develop EG-Reasoner, an evidence-aware model trained with explicit evidence supervision. EG-Reasoner achieves strong evidence grounding performance among open-source models while maintaining competitive answer generation results compared with proprietary systems. These findings highlight the importance of explicitly modeling evidence grounding and suggest that evidence-aware supervision provides an effective direction toward more reliable and interpretable VideoQA systems.

Benchmark Overview

Comparison between our EG-VQA and existing VideoQA benchmarks

Comparison between our EG-VQA and existing VideoQA benchmarks.

Overview of EG-VQA statistics and representative QA examples

Overview of EG-VQA statistics and representative QA examples. EG-VQA contains 2,067 videos and 11,838 QA pairs, split in a video-disjoint manner into 8,949 training and 2,889 test samples. It covers four reasoning categories: descriptive, temporal, causal, and counterfactual questions, progressing from directly observable video content to event ordering, causal inference, and hypothetical reasoning grounded in specific video segments.

The construction process of EG-VQA

Construction Pipeline. EG-VQA is constructed through a pipeline that combines video data curation, reasoning-oriented question-answer generation, prompt refinement, and two-stage quality control. Each QA pair is paired with temporally localized evidence.

Model

Overview of the EG-Reasoner training framework

EG-Reasoner Training Framework. Given a video and question, EG-Reasoner learns to produce structured responses that explicitly organize temporal evidence, intermediate reasoning, and the final answer. The model is optimized with reinforcement learning using a composite reward that jointly evaluates response format, answer correctness, and evidence grounding. During training, GRPO updates the policy by comparing multiple sampled responses, encouraging the model to generate faithful answers supported by localized video evidence.

Experiments

Experimental comparison of EG-Reasoner with commercial and open-source Video-LLMs

We compare the performance of EG-Reasoner with commercial large multimodal models and recent open-source Video-LLMs on EG-VQA. EG-Reasoner outperforms other open-source models in terms of both answer accuracy and evidence grounding.

Qualitative Analysis

Qualitative comparison between EG-Reasoner and the base model

As videos are analyzed, EG-Reasoner localizes key evidence regions and decomposes complex events into fine-grained temporal reasoning steps. Compared with the base model, it produces answers that are better grounded in the video content and less affected by loosely related context or hallucinated intermediate steps.

BibTeX

@misc{huang2026egvqa,
  title={EG-VQA: Benchmarking Verifiable Video Question Answering with Grounded Temporal Evidence},
  author={Huang, Linpeng and Chen, Weixing and Chen, Zexin and Liu, Yang and Lin, Liang},
  year={2026},
  eprint={2606.24797},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2606.24797}
}