关键词: 图神经网络, 视觉图神经网络, 视觉骨干网络, 图像分类, 目标检测, 实例分割

Abstract: Recently, Vision Graph Neural Networks (ViGs) have attracted considerable attention from the researchers in the field of computer vision, with graph construction being a key modeling approach in ViGs. The existing popular K-nearest neighbor (KNN) graph construction approach was limited by its fixed scale and quadratic computational complexity, making it difficult to model both local and multi-scale information in an image. To address this issue, a Multi-Scale Sparse Graph (MSSG) construction method was proposed. MSSG decomposes the KNN graph into three sparse subgraphs of different scales along the channel dimension, achieving linear computational complexity while effectively modeling both local and multi-scale information in an image. To enhance the model's global modeling capability, a global and local multi-scale information fusion strategy was proposed. Based on these innovations, a novel vision backbone network, termed MSViG, was proposed. The image classification experiments on ImageNet-1K demonstrate that the proposed vision architecture outperforms the existing Vision Graph Neural Networks. For example, MSViG-T achieves a 2.1 percentage points higher Top-1 classification accuracy compared to ViG-T and shows the significant performance improvements in downstream vision tasks such as object detection and instance segmentation compared to ViGs.

Key words: graph neural network, vision graph neural network, vision backbone, image classification, object detection, instance segmentation