关键词: 视频问答, 多模态大语言模型, 关键帧定位, 多模态推理, 视频摘要

Abstract:

The video question answering task aims to understand video content deeply and answer natural language questions. Current methods based on Multimodal Large Language Models （MLLMs） struggle to model global semantics and temporal dependencies across scenes effectively due to dual constraints of context length and computational complexity， resulting in degraded model inference performance. To address this issue， a Scene-Driven Adaptive Keyframe Sampling （SD-AKS） method and a Question-Driven Video Summarization （QD-VS） method were proposed. First， an iterative video scene clustering based on K-means was introduced， and a Scene Separation Score （SSS） was designed， so as to evaluate feature differences between scenes quantitatively， thereby enhancing the accuracy and robustness of scene segmentation. Second， a problem semantic-guided adaptive keyframe sampling was designed to achieve dynamic keyframe selection based on quantified scene information density and fine coverage of information-dense regions. Furthermore， based on the keyframe method， Large Language Models （LLMs） were employed to summarize the video-related question set， so that the task-oriented textual summaries were generated to improve global model inference capability. Experimental results show that compared to LLaVA?Video， the best?performing baseline model， the proposed method improves prediction accuracy by 5.3 and 1.8 percentage points on the EgoSchema and NExT?QA datasets， respectively， which verifies its effectiveness in cross?dataset scenes.

Key words: video question answering, Multimodal Large Language Model (MLLM), keyframe localization, multimodal reasoning, video summarization