Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (6): 1816-1823.DOI: 10.11772/j.issn.1001-9081.2023060811

Special Issue: 人工智能

• Artificial intelligence • Previous Articles     Next Articles

Semi-supervised heterophilic graph representation learning model based on Graph Transformer

Shibin LI1, Jun GONG1(), Shengjun TANG2   

  1. 1.School of Software,Jiangxi Normal University,Nanchang Jiangxi 330022,China
    2.School of Architecture & Urban Planning,Shenzhen University,Shenzhen Guangdong 518060,China
  • Received:2023-06-25 Revised:2023-09-11 Accepted:2023-09-12 Online:2023-10-07 Published:2024-06-10
  • Contact: Jun GONG
  • About author:LI Shibin, born in 1999, M. S. candidate. His research interests include graph deep learning, complex network.
    TANG Shengjun, born in 1991, Ph. D., assistant professor. His research interests include robotic SLAM.
  • Supported by:
    Natural Science Foundation of Guangdong Province(2121A1515012574)

基于Graph Transformer的半监督异配图表示学习模型

黎施彬1, 龚俊1(), 汤圣君2   

  1. 1.江西师范大学 软件学院,南昌 330022
    2.深圳大学 建筑与城市规划学院,广东 深圳 518060
  • 通讯作者: 龚俊
  • 作者简介:黎施彬(1999—),男,重庆人,硕士研究生,主要研究方向:图深度学习、复杂网络
    汤圣君(1991—),男,江西抚州人,助理教授,博士,主要研究方向:机器人SLAM。
  • 基金资助:
    广东省自然科学基金资助项目(2121A1515012574)

Abstract:

Existing Graph Convolutional Network (GCN) methods are based on the assumption of homophily, which cannot be directly applied to heterophilic graph representation learning, and many studies on heterophilic graph representation learning are limited by message-passing mechanism, which leads to the problem of over-smoothing due to the confusion and over-squeezing of node features. To address these issues, a semi-supervised heterophilic graph representation learning model based on Graph Transformer,named HPGT(HeteroPhilic Graph Transformer), was proposed. Firstly, the path neighborhood of a node was sampled using the degree connection probability matrix, then the heterophilic connection patterns of nodes on the path were adaptively aggregated through the self-attention mechanism, which were encoded to obtain the structural information of nodes, and the original attribute information and structural information of nodes were used to construct the self-attention module of the Transformer layer. Secondly, the hidden layer representation of each node itself was separated from those of its neighboring nodes and updated to avoid the node aggregating too much information about itself through the self-attention module, and then the representation and the neighborhood representation of nodes were connected to get the output of a single Transformer layer; in addition, the outputs of all Transformer layers were connected to get the final node hidden layer representation so as to prevent the loss of information in middle layers. Finally, the linear layer and Softmax layer were used to map the hidden layer representations of nodes to the predictive labels of nodes. In the comparison experiments with the model without Structural Encoding (SE), SE based on degree connection probability provides effective deviation information for self-attention modules of Transformer layers, and improves the average accuracy of HPGT by 0.99% to 11.98%. Compared with the comparative models, on the heterophilic datasets (Texas, Cornell, Wisconsin, and Actor), the node classification accuracies of HPGT are improved by 0.21% to 1.69%, and on homophilic datasets (Cora, CiteSeer, and PubMed), the node classification accuracies reach 0.837 9, 0.746 7 and 0.886 2, respectively. The experimental results show that HPGT has a strong ability for heterogeneous graph representation learning, and is particularly suitable for node classification tasks of strong heterophilic graphs.

Key words: Graph Convolutional Network (GCN), heterophilic graph, graph representation learning, Graph Transformer, node classification

摘要:

现有的图卷积网络(GCN)模型基于同配性假设,无法直接应用于异配图的表示学习,且许多异配图表示学习的研究工作受消息传递机制的限制,导致节点特征混淆和特征过度挤压而出现过平滑问题。针对这些问题,提出一种基于Graph Transformer的半监督异配图表示学习模型HPGT(HeteroPhilic Graph Transformer)。首先,使用度连接概率矩阵采样节点的路径邻域,再通过自注意力机制自适应地聚合路径上的节点异配连接模式,编码得到节点的结构信息,用节点的原始属性信息和结构信息构建Transformer层的自注意力模块;其次,将每个节点自身的隐层表示与它的邻域节点的隐层表示分离更新以避免节点通过自注意力模块聚合过量的自身信息,再把每个节点表示与它的邻域表示连接,得到单个Transformer层的输出,另外,将所有的Transformer层的输出跳连到最终的节点隐层表示以防止中间层信息丢失;最后,使用线性层和Softmax层将节点的隐层表示映射到节点的预测标签。实验结果表明,与无结构编码(SE)的模型相比,基于度连接概率的SE能为Transformer层的自注意力模块提供有效的偏差信息,HPGT平均准确率提升0.99%~11.98%;与对比模型相比,在异配数据集(Texas、Cornell、Wisconsin和Actor)上,模型节点分类准确率提升0.21%~1.69%,在同配数据集(Cora、CiteSeer和PubMed)上,节点分类准确率分别达到了0.837 9、0.746 7和0.886 2。以上结果验证了HPGT具有较强的异配图表示学习能力,尤其适用于强异配图节点分类任务。

关键词: 图卷积网络, 异配图, 图表示学习, Graph Transformer, 节点分类

CLC Number: