Adaptive deep graph convolution using initial residual and decoupling operations

doi:10.11772/j.issn.1001-9081.2021071289

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (1): 9-15.DOI: 10.11772/j.issn.1001-9081.2021071289

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Adaptive deep graph convolution using initial residual and decoupling operations

Jijie ZHANG¹, Yan YANG¹^,²(), Yong LIU¹^,²

^1.School of Computer Science and Technology，Heilongjiang University，Harbin Heilongjiang 150080，China
^2.Key Laboratory of Database and Parallel Computing of Heilongjiang Province （Heilongjiang University），Harbin Heilongjiang 150080，China

Received:2021-07-19 Revised:2021-08-13 Accepted:2021-08-19 Online:2021-08-13 Published:2022-01-10
Contact: Yan YANG
About author:ZHANG Jijie， born in 1998， M. S. candidate. His research interests include graph representation learning， recommendation system.
YANG Yan， born in 1973， Ph. D.， professor. Her research interests include data mining， social network.
LIU Yong， born in 1975， Ph. D.， associate professor. His research interests include data mining， machine learning.
Supported by:
Natural Science Foundation of Heilongjiang Province(LH2020F043)

利用初始残差和解耦操作的自适应深层图卷积

张继杰¹, 杨艳¹^,²(), 刘勇¹^,²

^1.黑龙江大学计算机科学技术学院，哈尔滨 150080
^2.黑龙江省数据库与并行计算重点实验室（黑龙江大学），哈尔滨 150080

通讯作者: 杨艳
作者简介:张继杰（1998—），男，山东青岛人，硕士研究生，CCF会员，主要研究方向：图表示学习、推荐系统
杨艳（1973—），女，黑龙江哈尔滨人，教授，博士，CCF会员，主要研究方向：数据挖掘、社会网
刘勇（1975—），男，河北昌黎人，副教授，博士，CCF会员，主要研究方向：数据挖掘、机器学习。

Abstract

Abstract:

The traditional Graph Convolutional Network （GCN） and many of its variants achieve the best effect in the shallow layers， and do not make full use of higher-order neighbor information of nodes in the graph. The subsequent deep graph convolution models can solve the above problem， but inevitably generate the problem of over-smoothing， which makes the models impossible to effectively distinguish different types of nodes in the graph. To address this problem， an adaptive deep graph convolution model using initial residual and decoupling operations， named ID-AGCN （model using Initial residual and Decoupled Adaptive Graph Convolutional Network）， was proposed. Firstly， the node’s representation transformation as well as feature propagation was decoupled. Then， the initial residual was added to the node’s feature propagation process. Finally， the node representations obtained from different propagation layers were combined adaptively， appropriate local and global information was selected for each node to obtain node representations containing rich information， and a small number of labeled nodes were used for supervised training to generate the final node representations. Experimental result on three datasets Cora， CiteSeer and PubMed indicate that the classification accuracy of ID-AGCN is improved by about 3.4 percentage points， 2.3 percentage points and 1.9 percentage points respectively， compared with GCN. The proposed model has superiority in alleviating over-smoothing.

Key words: node classification, initial residual, decoupling, adaptive, Graph Convolutional Network (GCN)

摘要：

传统的图卷积网络（GCN）及其很多变体都是在浅层时达到最佳的效果，而没有充分利用图中节点的高阶邻居信息。随后产生的深层图卷积模型可以解决以上问题却又不可避免地产生了过平滑的问题，导致模型无法有效区分图中不同类别的节点。针对此问题，提出了一种利用初始残差和解耦操作的自适应深层图卷积模型ID-AGCN。首先，对节点的表示转换以及特征传播进行解耦；然后，在节点的特征传播过程中添加了初始残差；最后，自适应地结合不同传播层得到的节点表示，针对每个节点选择其合适的局部信息和全局信息以得到含有丰富信息的节点表征，并利用少部分带标签的节点进行监督训练来生成最终的节点表征。在Cora、CiteSeer和PubMed这三个数据集上的实验结果表明，ID-AGCN的分类准确率相较GCN分别提高了约3.4个百分点、2.3个百分点和1.9个百分点。所提模型能够更好地缓解过平滑。

关键词: 节点分类, 初始残差, 解耦, 自适应, 图卷积网络

CLC Number:

TP183

Jijie ZHANG, Yan YANG, Yong LIU. Adaptive deep graph convolution using initial residual and decoupling operations[J]. Journal of Computer Applications, 2022, 42(1): 9-15.

张继杰, 杨艳, 刘勇. 利用初始残差和解耦操作的自适应深层图卷积[J]. 《计算机应用》唯一官方网站, 2022, 42(1): 9-15.

Figures/Tables 11

Tab. 1 Symbols and their definition

符号	含义
$G = (V, E)$	节点集 $V$ 、边集 $E$ 组成的图
$V$	图中节点的集合
$E$	图中边的集合
$X ∈ R N × d$	初始的节点特征矩阵
$Y ∈ R N × c$	标签矩阵
$Z ∈ R N × c$	用于预测的节点表示矩阵
MLP	多层感知机
$d$	初始的节点特征维度
$k$	图卷积层数
$α$	残差保留率
$c$	节点类别数
$N (v i)$	节点 $v i$ 的邻居集合

Tab. 1 Symbols and their definition

符号	含义
$G = (V, E)$	节点集 $V$ 、边集 $E$ 组成的图
$V$	图中节点的集合
$E$	图中边的集合
$X ∈ R N × d$	初始的节点特征矩阵
$Y ∈ R N × c$	标签矩阵
$Z ∈ R N × c$	用于预测的节点表示矩阵
MLP	多层感知机
$d$	初始的节点特征维度
$k$	图卷积层数
$α$	残差保留率
$c$	节点类别数
$N (v i)$	节点 $v i$ 的邻居集合

Fig. 1 Framework of the proposed model

Tab. 2 Citation datasets

数据集	节点数	边数	训练节点数	验证节点数	测试节点数	节点类别数	特征维度	边密度
Cora	2 708	5 278	140	500	1 000	7	1 433	0.001 4
CiteSeer	3 327	4 552	120	500	1 000	6	3 703	0.000 8
PubMed	19 717	44 324	60	500	1 000	3	500	0.000 2

Tab. 3 Classification accuracy results for citation datasets

模型	Cora	CiteSeer	PubMed
ChebNet	80.6±1.1	70.0±1.3	78.1±0.6
GCN	81.3±0.6	71.1±0.8	78.9±0.5
GAT	83.1±0.4	70.8±0.5	79.0±0.3
SGC	81.7±0.1	71.3±0.3	78.9±0.0
APPNP	83.2±0.4	71.8±0.5	80.2±0.3
DAGNN	84.4±0.5	73.3±0.6	80.5±0.5
ID-AGCN	84.7±0.6	73.4±0.6	80.8±0.4

Tab. 4 Parameter setting

数据集

残差

保留率 $α$

图卷积

层数 $k$

Tab. 4 Parameter setting

数据集

残差

保留率 $α$

图卷积

层数 $k$

weight decay

dropout rate

学习率

Cora

0.02

0.005

0.85

0.01

CiteSeer

0.05

0.020

0.55

0.01

PubMed

0.01

0.010

0.85

0.01

Fig. 2 Comparison of over-smoothing alleviation results

Tab. 5 Classification accuracy and running time comparison of different models

模型	最优层数	分类准确率/%	运行时间/s
ChebNet	3	80.6±1.1	2.57
GCN	2	81.3±0.6	1.85
GAT	2	83.1±0.4	7.38
SGC	3	81.7±0.1	1.50
APPNP	10	83.2±0.4	1.61
DAGNN	10	84.4±0.5	4.31
ID-AGCN	15	84.7±0.6	4.48

Fig. 3 Result comparison of ablation experiments

Fig. 4 Influence of parameter α

Fig. 5 Influence of parameter k

Fig. 6 Cora dataset visualization results

References 23

1	WU L， SUN P J， FU Y J， et al. A neural influence diffusion model for social recommendation［C］// Proceedings of the 42nd International Conference on Research and Development in Information Retrieval. New York： ACM， 2019： 235-244. 10.1145/3331184.3331214
2	FAN W Q， MA Y， LI Q， et al. Graph neural networks for social recommendation［C］// Proceedings of the 2019 World Wide Web Conference. New York： ACM， 2019： 417-426. 10.1145/3308558.3313488
3	WANG H W， ZHAO M， XIE X， et al. Knowledge graph convolutional networks for recommender systems［C］// Proceedings of the 2019 World Wide Web Conference. New York： ACM， 2019： 3307-3313. 10.1145/3308558.3313417
4	JIN W G， COLEY C W， BARZILAY R， et al. Predicting organic reaction outcomes with Weisfeiler-Lehman network［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 2607-2616.
5	KIPF T N， WELLING M. Semi-supervised classification with graph convolutional networks［EB/OL］. （2017-02-22）［2021-01-20］. .
6	VELIČKOVIĆ P， CUCURULL G， CASANOVA A， et al. Graph attention networks［EB/OL］. （2018-02-04）［2021-02-10］..
7	涂存超，杨成，刘知远，等. 网络表示学习综述［J］. 中国科学：信息科学， 2017， 47（8）： 980-996. 10.1360/n112017-00145
	TU C C， YANG C， LIU Z Y， et al. Network representation learning： an overview［J］. SCIENTIA SINICA Informationis， 2017， 47（8）： 980-996. 10.1360/n112017-00145
8	LI Q M， HAN Z C， WU X M. Deeper insights into graph convolutional networks for semi-supervised learning［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2018： 3538-3545.
9	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
10	PAGE L， BRIN S， MOTWANI R， et al. The PageRank citation ranking： bring order to the Web［R］. Stanford： Stanford InfoLab， 1999.
11	KLICPERA J， BOJCHEVSKI A， GÜNNEMANN S. Predict then propagate： graph neural networks meet personalized PageRank［EB/OL］. （2019-02-27）［2021-02-20］.. 10.1145/3394486.3403296
12	CHIANG W L， LIU X Q， SI S， et al. Cluster-GCN： an efficient algorithm for training deep and large graph convolutional networks［C］// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2019： 257-266. 10.1145/3292500.3330925
13	CHEN M， WEI Z W， HUANG Z F， et al. Simple and deep graph convolutional networks［C］// Proceedings of the 37th International Conference on Machine Learning. New York： JMLR.org， 2020： 1725-1735.
14	IOFFE S， SZEGEDY C. Batch normalization： accelerating deep network training by reducing internal covariate shift［C］// Proceedings of the 32nd International Conference on Machine Learning. New York： JMLR.org， 2015： 448-456.
15	ZHAO L X， AKOGLU L. PairNorm： tackling over-smoothing in GNNs［EB/OL］. （2020-02-13）［2021-02-25］..
16	RONG Y， HUANG W B， XU T Y， et al. DropEdge： towards deep graph convolutional networks on node classification［EB/OL］. （2020-03-12）［2021-03-05］. . 10.1007/978-3-030-75765-6_6
17	SRIVASTAVA N， HINTON G， KRIZHEVSKY A， et al. Dropout： a simple way to prevent neural networks from overfitting［J］. Journal of Machine Learning Research， 2014， 15： 1929-1958.
18	FENG W Z， ZHANG J， DONG Y X， et al. Graph random neural networks for semi-supervised learning on graphs［C/OL］// Proceedings of the 34th International Conference on Neural Information Processing Systems. （2020-10-26）［2021-03-05］..
19	LIU M， GAO H Y， JI S W. Towards deeper graph neural networks［C］// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2020： 338-348. 10.1145/3394486.3403076
20	WU F， ZHANG T Y， DE SOUZA A H， et al. Simplifying graph convolutional networks［C］// Proceedings of the 36th International Conference on Machine Learning. New York： JMLR.org， 2019： 6861-6871.
21	张璞，柴变芳，张静，等. 半监督属性网络表示学习方法［J］. 计算机工程与应用， 2019， 55（12）： 117-123， 144. 10.3778/j.issn.1002-8331.1812-0079
	ZHANG P， CHAI B F， ZHANG J， et al. Semi-supervised representation learning method for attributed networks［J］. Computer Engineering and Applications， 2019， 55（12）： 117-123， 144. 10.3778/j.issn.1002-8331.1812-0079
22	YANG Z L， COHEN W， SALAKHUTDINOV R. Revisiting semi-supervised learning with graph embeddings［C］// Proceedings of the 33rd International Conference on Machine Learning. New York： JMLR.org， 2016： 40-48.
23	DEFFERRARD M， BRESSON X， VANDERGHEYNST P. Convolutional neural networks on graphs with fast localized spectral［C］// Proceedings of the 30th International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2016： 3844-3852.

[1]	Qinzhuang ZHAO, Hongye TAN. Time series causal inference method based on adaptive threshold learning [J]. Journal of Computer Applications, 2024, 44(9): 2660-2666.
[2]	Jiepo FANG, Chongben TAO. Hybrid internet of vehicles intrusion detection system for zero-day attacks [J]. Journal of Computer Applications, 2024, 44(9): 2763-2769.
[3]	Guanglei YAO, Juxia XIONG, Guowu YANG. Flower pollination algorithm based on neural network optimization [J]. Journal of Computer Applications, 2024, 44(9): 2829-2837.
[4]	Chuanlin PANG, Rui TANG, Ruizhi ZHANG, Chuan LIU, Jia LIU, Shibo YUE. Distributed power allocation algorithm based on graph convolutional network for D2D communication systems [J]. Journal of Computer Applications, 2024, 44(9): 2855-2862.
[5]	Jinjin LI, Guoming SANG, Yijia ZHANG. Multi-domain fake news detection model enhanced by APK-CNN and Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2674-2682.
[6]	Guixiang XUE, Hui WANG, Weifeng ZHOU, Yu LIU, Yan LI. Port traffic flow prediction based on knowledge graph and spatio-temporal diffusion graph convolutional network [J]. Journal of Computer Applications, 2024, 44(9): 2952-2957.
[7]	Le YANG, Damin ZHANG, Qing HE, Jiaxin DENG, Fengqin ZUO. Application of improved hunter-prey optimization algorithm in WSN coverage [J]. Journal of Computer Applications, 2024, 44(8): 2506-2513.
[8]	Hang XU, Zhi YANG, Xingyuan CHEN, Bing HAN, Xuehui DU. Coverage-guided fuzzing based on adaptive sensitive region mutation [J]. Journal of Computer Applications, 2024, 44(8): 2528-2535.
[9]	Yuwei DING, Hongbo SHI, Jie LI, Min LIANG. Image denoising network based on local and global feature decoupling [J]. Journal of Computer Applications, 2024, 44(8): 2571-2579.
[10]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[11]	Yan LI, Dazhi PAN, Siqing ZHENG. Improved adaptive large neighborhood search algorithm for multi-depot vehicle routing problem with time window [J]. Journal of Computer Applications, 2024, 44(6): 1897-1904.
[12]	Jinfu WU, Yi LIU. Fast adversarial training method based on random noise and adaptive step size [J]. Journal of Computer Applications, 2024, 44(6): 1807-1815.
[13]	Chao GE, Jiabin ZHANG, Lei WANG, Zhixin LUN. Trajectory planning for autonomous vehicles based on model predictive control [J]. Journal of Computer Applications, 2024, 44(6): 1959-1964.
[14]	Shibin LI, Jun GONG, Shengjun TANG. Semi-supervised heterophilic graph representation learning model based on Graph Transformer [J]. Journal of Computer Applications, 2024, 44(6): 1816-1823.
[15]	Wenping ZHENG, Huilin GE, Meilin LIU, Gui YANG. Node classification algorithm fusing 2-connected motif-structure information [J]. Journal of Computer Applications, 2024, 44(5): 1464-1470.

Adaptive deep graph convolution using initial residual and decoupling operations

利用初始残差和解耦操作的自适应深层图卷积

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 23

Related Articles 15

Recommended Articles

Metrics