关键词: 图像中文描述, 图像多级视觉特征, 多粒度, 图文交互, 动态融合

Abstract: Image caption technology can help computers understand image content better, and achieve cross-modal interaction. To address the issues of incomplete extraction of multi-scale features from images and insufficient understanding of image-text correlation in Chinese image caption tasks, this paper proposed a method that extracted multi-level visual and semantic features of images and dynamically integrated them in the decoding process. First, multi-level visual features were extracted on the encoder side, with fine-grained features obtained through an auxiliary guidance module for local feature extraction. Then, a text-image interation module was designed to dynamically focus on semantic associations between image and text information. Finally, a dynamic feature fusion decoder integrated and decoded features with dynamically weighted image-text information through a closed-loop fusion and attention mechanism, ensuring enhanced information without loss, to generate semantically coherent Chinese image descriptions. Use BLEU-n, Rouge, Meteor, and CIDEr evaluation indicators for method evaluation, and compare this method with six different methods, the score indicators of semantic relevance are improved. Compared with the baseline model, BLEU-1, BLEU-2, BLEU-3, BLEU-4, Rouge_L, Meteor, and CIDEr improved by 5.32%, 6.76%, 8.07%, 9.78%, 12.33%, 4.88%, and 13.16% respectively. It shows that this method has better accuracy.

Key words: Chinese image caption, image multi-level visual features, multi-granularity, image-text interaction, dynamic fusion