Attribute network representation learning with dual auto-encoder

doi:10.11772/j.issn.1001-9081.2022091337

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (8): 2338-2344.DOI: 10.11772/j.issn.1001-9081.2022091337

• The 19th International Conference on Web Information Systems and Applications (WISA 2022) • Previous Articles Next Articles

Attribute network representation learning with dual auto-encoder

Jinghong WANG¹^,²^,³, Zhixia ZHOU¹, Hui WANG⁴(), Haokang LI⁴

^1.College of Computer and Cyber Security，Hebei Normal University，Shijiazhuang Hebei 050024，China
^2.Hebei Provincial Key Laboratory of Network and Information Security （Hebei Normal University），Shijiazhuang Hebei 050024，China
^3.Hebei Provincial Engineering Research Center for Supply Chain Big Data Analytics and Security （Hebei Normal University），Shijiazhuang Hebei 050024，China
^4.Hebei Polytechnic Institute，Shijiazhuang Hebei 050091，China

Received:2022-09-06 Revised:2022-09-27 Accepted:2022-10-08 Online:2022-10-13 Published:2023-08-10
Contact: Hui WANG
About author:WANG Jinghong， born in 1967， Ph. D.， professor. Her research interests include artificial intelligence， big data， data mining.
ZHOU Zhixia， born in 1996， M. S. candidate. Her research interests include data mining， network representation learning.
LI Haokang， born in 1994， M. S.His research interests include community discovery， deep learning， graph representation learning.
Supported by:
Central Guidance on Local Science and Technology Development Fund of Hebei Province(226Z1808G);Hebei Natural Science Foundation(F2021205014);Science and Technology Project of Hebei Colleges and Universities(ZD2022139);Hebei Normal University Science and Technology Major Project(L2023J05)

双路自编码器的属性网络表示学习

王静红¹^,²^,³, 周志霞¹, 王辉⁴(), 李昊康⁴

^1.河北师范大学计算机与网络空间安全学院，石家庄 050024
^2.河北省网络与信息安全重点实验室（河北师范大学），石家庄 050024
^3.供应链大数据分析与数据安全河北省工程研究中心（河北师范大学），石家庄 050024
^4.河北工程技术学院，石家庄 050091

通讯作者: 王辉
作者简介:王静红（1967—），女，河北石家庄人，教授，博士，CCF会员，主要研究方向：人工智能、大数据、数据挖掘
周志霞（1996—），女，河北石家庄人，硕士研究生，CCF会员，主要研究方向：数据挖掘、网络表示学习
李昊康（1994—），男，河北石家庄人，硕士，CCF会员，主要研究方向：社区发现、深度学习、图表示学习。
基金资助:
中央引导地方科技发展资金资助项目(226Z1808G);河北省自然科学基金资助项目(F2021205014);河北省高等学校科学技术研究项目(ZD2022139);河北师范大学重点项目(L2023J05)

Abstract

Abstract:

On the premise of ensuring the properties of nodes in the network， the purpose of attribute network representation learning is to learn the low-dimensional dense vector representation of nodes by combining structure and attribute information. In the existing attribute network representation learning methods， the learning of attribute information in the network is ignored， and the interaction of attribute information with the network topology is insufficient， so that the network structure and attribute information cannot be fused efficiently. In response to the above problems， a Dual auto-Encoder Network Representation Learning （DENRL） algorithm was proposed. Firstly， the high-order neighborhood information of nodes was captured through a multi-hop attention mechanism. Secondly， a low-pass Laplacian filter was designed to remove the high-frequency signals and iteratively obtain the attribute information of important neighbor nodes. Finally， an adaptive fusion module was constructed to increase the acquisition of important information through the consistency and difference constraints of the two kinds of information， and the encoder was trained by supervising the joint reconstruction loss function of the two auto-encoders. Experimental results on Cora， Citeseer， Pubmed and Wiki datasets show that DENRL algorithm has the highest clustering accuracy and the lowest algorithm running time on three citation network datasets compared with DeepWalk， ANRL （Attributed Network Representation Learning） and other algorithms， achieves these two indicators of 0.775 and 0.460 2 s respectively on Cora datasets， and has the highest link prediction precision on Cora and Citeseer datasets， reaching 0.961 and 0.970 respectively. It can be seen that the fusion and interactive learning of attribute and structure information can obtain stronger node representation capability.

Key words: attribute network, network representation learning, auto-encoder, interactive learning, attention mechanism

摘要：

属性网络表示学习的目的是在保证网络中节点性质的前提下，结合结构和属性信息学习节点的低维稠密向量表示。目前属性网络表示学习方法忽略了网络中属性信息的学习，且这些方法中的属性信息与网络拓扑结构的交互性不足，不能高效融合网络结构和属性信息。针对以上问题，提出一种双路自编码器的属性网络表示学习（DENRL）算法。首先，通过多跳注意力机制捕获节点的高阶邻域信息；其次，设计低通拉普拉斯滤波器去除高频信号，并迭代获取重要邻居节点的属性信息；最后，构建自适应融合模块，通过结构和属性信息的一致性及差异性约束来增加对重要信息的获取，并通过监督两个自编码器的联合重构损失函数训练编码器。在Cora、Citeseer、Pubmed和Wiki数据集上的实验结果表明，与DeepWalk、ANRL（Attributed Network Representation Learning）等算法相比，DENRL算法在3个引文网络数据集上聚类准确率最高、算法运行时间最少，在Cora数据集上聚类准确率为0.775和运行时间为0.460 2 s；且DENRL算法在Cora和Citeseer数据集上链路预测精确率最高，分别达到了0.961和0.970。可见，属性与结构信息的融合及交互学习可以获得更强的节点表示能力。

关键词: 属性网络, 网络表示学习, 自编码器, 交互学习, 注意力机制

CLC Number:

TP181

Jinghong WANG, Zhixia ZHOU, Hui WANG, Haokang LI. Attribute network representation learning with dual auto-encoder[J]. Journal of Computer Applications, 2023, 43(8): 2338-2344.

王静红, 周志霞, 王辉, 李昊康. 双路自编码器的属性网络表示学习[J]. 《计算机应用》唯一官方网站, 2023, 43(8): 2338-2344.

Figures/Tables 9

Tab. 1 Symbol meaning

符号	含义
$V$	节点集合
$E$	节点之间边的集合
$A$	节点属性集合
$n$	网络节点数，即 $\| V \|$
$q$	节点属性数，即 $\| A \|$
$X$	属性矩阵，大小为 $n × q$
$M$	邻接矩阵
$e i j$	节点 $v i$ 与 $v j$ 之间权重
$v i$	标号为 $i$ 的节点， $v i ∈ V$
$d$	节点最终表示向量的维度， $d ≫ t$
$y i$	节点 $v i$ 的表示向量
$Y$	节点表示向量矩阵
$y i x$	属性自编码器生成的节点 $v i$ 属性嵌入表示
$y i m$	结构自编码器生成的节点 $v i$ 结构嵌入表示

Tab. 1 Symbol meaning

符号	含义
$V$	节点集合
$E$	节点之间边的集合
$A$	节点属性集合
$n$	网络节点数，即 $\| V \|$
$q$	节点属性数，即 $\| A \|$
$X$	属性矩阵，大小为 $n × q$
$M$	邻接矩阵
$e i j$	节点 $v i$ 与 $v j$ 之间权重
$v i$	标号为 $i$ 的节点， $v i ∈ V$
$d$	节点最终表示向量的维度， $d ≫ t$
$y i$	节点 $v i$ 的表示向量
$Y$	节点表示向量矩阵
$y i x$	属性自编码器生成的节点 $v i$ 属性嵌入表示
$y i m$	结构自编码器生成的节点 $v i$ 结构嵌入表示

Fig.1 Framework of dual auto-encoder

Tab. 2 Statistics of datasets

数据集	节点数	边数	属性数	标签数
Citeseer	3 312	4 714	3 703	6
Pubmed	19 717	44 338	500	3
Cora	2 708	5 429	1 433	7
Wiki	2 405	17 981	4 973	17

Tab. 3 Parameter setting of different datasets

数据集	t	lr	数据集	t	lr
Cora	8	$1 × 10 - 3$	Pubmed	35	$1 × 10 - 4$
Citeseer	3	$3 × 10 - 3$	Wiki	1	$1 × 10 - 3$

Tab. 3 Parameter setting of different datasets

数据集	t	lr	数据集	t	lr
Cora	8	$1 × 10 - 3$	Pubmed	35	$1 × 10 - 4$
Citeseer	3	$3 × 10 - 3$	Wiki	1	$1 × 10 - 3$

Tab. 4 Experimental results of node clustering

算法	Cora		Citeseer		Pubmed		Wiki
算法	ACC	NMI	ACC	NMI	ACC	NMI	ACC	NMI
DeepWalk^［7］	0.482	0.328	0.326	0.088	0.543	0.105	0.388	0.223
node2vec^［8］	0.647	0.356	0.451	0.101	0.664	0.127	0.379	—
LINE^［10］	0.479	0.433	0.391	0.225	0.661	0.387	0.409	—
TADW^［12］	0.599	0.443	0.455	0.290	0.511	0.244	0.311	0.118
DANE^［24］	0.702	0.630	0.479	0.422	0.694	0.308	0.473	0.499
AANE^［13］	0.445	0.161	0.447	0.143	0.451	—	0.432	—
VAE^［15］	0.616	0.490	0.367	0.223	0.631	0.248	0.377	0.374
VGAE^［16］	0.554	0.407	0.377	0.281	0.627	0.333	0.444	0.299
ANRL^［19］	0.597	0.431	0.522	0.399	0.469	0.305	0.426	0.344
DENRL	0.775	0.695	0.705	0.458	0.709	0.326	0.468	0.497

Fig.2 Experimental results of NMI comparison for different k values

Tab. 5 Comparison of average running time between DENRL algorithm and other algorithms

算法	Cora	Citeseer	Pubmed	Wiki
DeepWalk	0.629 8	1.263 8	32.746 9	1.499 7
TADW	0.854 6	1.637 6	30.587 5	53.348 6
VAE	0.555 4	0.997 8	22.904 5	17.256 7
VGAE	0.506 3	0.905 6	20.000 6	16.497 6
DENRL	0.4602	0.8547	17.4906	18.6905

Tab. 6 Comparison of AUC and AP for link prediction

算法	Cora		Citeseer
算法	AUC	AP	AUC	AP
DeepWalk	0.803	0.817	0.732	0.761
TADW	0.931	0.939	0.945	0.957
DANE	0.882	0.895	0.848	0.846
VAE	0.910	0.921	0.892	0.898
VGAE	0.914	0.926	0.909	0.901
DENRL	0.955	0.961	0.968	0.970

Tab. 7 Comparison of ablation experiment results

模型	ACC
模型	Cora	Citeseer	Pubmed
Structure-only（M）	0.666	0.699	0.506
Attribute-only（X）	0.753	0.692	0.673
Str+Attribute（M+X）	0.775	0.705	0.709

References 25

1	ZHOU J Y， LIU L， WEI W Q， et al. Network representation learning： from preprocessing， feature extraction to node embedding［J］. ACM Computing Surveys， 2023， 55（2）： No.38. 10.1145/3491206
2	AMARA A， TAIEB M A H， AOUICHA M B. Network representation learning systematic review： ancestors and current development state［J］. Machine Learning with Applications， 2021， 6： No.100130. 10.1016/j.mlwa.2021.100130
3	SUN H L， HE F， HUANG J B， et al. Network embedding for community detection in attributed networks［J］. ACM Transactions on Knowledge Discovery from Data， 2020， 14（3）： No.36. 10.1145/3385415
4	XU M J. Understanding graph embedding methods and their applications［J］. SIAM Review， 2021， 63（4）： 825-853. 10.1137/20m1386062
5	TONG N， TANG Y， CHEN B， et al. Representation learning using attention network and CNN for heterogeneous networks［J］. Expert Systems with Applications， 2021， 185： No.115628. 10.1016/j.eswa.2021.115628
6	LIAO L Z， HE X N， ZHANG H W， et al. Attributed social network embedding［J］. IEEE Transactions on Knowledge and Data Engineering， 2018， 30（12）： 2257-2270. 10.1109/tkde.2018.2819980
7	PEROZZI B， AL-RFOU R， SKIENA S. DeepWalk： online learning of social representations［C］// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2014： 701-710. 10.1145/2623330.2623732
8	GROVER A， LESKOVEC J. node2vec： Scalable feature learning for networks［C］// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2016： 855-864. 10.1145/2939672.2939754
9	RIBEIRO L F R， SAVERESE P H P， FIGUEIREDO D R. struc2vec： Learning node representations from structural identity［C］// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2017： 385-394. 10.1145/3097983.3098061
10	TANG J， QU M， WANG M Z， et al. LINE： large-scale information network embedding［C］// Proceedings of the 24th International Conference on World Wide Web. Republic and Canton of Geneva： International World Wide Web Conferences Steering Committee， 2015： 1067-1077. 10.1145/2736277.2741093
11	WANG D X， CUI P， ZHU W W. Structural deep network embedding［C］// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2016： 1225-1234. 10.1145/2939672.2939753
12	YANG C， LIU Z Y， ZHAO D L， et al. Network representation learning with rich text information［C］// Proceedings of the 24th International Joint Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2015： 2111-2117. 10.1609/aaai.v29i1.9448
13	HUANG X， LI J D， XU X. Accelerated attributed network embedding［C］// Proceedings of the 2017 SIAM International Conference on Data Mining. Philadelphia， PA： SIAM， 2017： 633-641. 10.1137/1.9781611974973.71
14	ZHAO Z Y， ZHOU H， LI C， et al. DeepEmLAN： deep embedding learning for attributed networks［J］. Information Sciences， 2021， 543： 382-397. 10.1016/j.ins.2020.07.001
15	KINGMA D P， WELLING M. Auto-encoding variational Bayes［EB/OL］. （2022-12-10）［2023-03-21］.. 10.1561/2200000056
16	KIPF T N， WELLING M. Variational graph auto-encoders［EB/OL］. （2016-11-21）［2022-04-22］..
17	HAMILTON W L， YING R， LESKOVEC J. Inductive representation learning on large graphs［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 1025-1035. 10.7551/mitpress/11474.003.0014
18	HUANG X， SONG Q Q， LI Y N， et al. Graph recurrent networks with attributed random walks［C］// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2019： 732-740. 10.1145/3292500.3330941
19	ZHANG Z， YANG H X， BU J J， et al. ANRL： attributed network representation learning via deep neural networks［C］// Proceedings of the 27th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2018： 3155-3161. 10.24963/ijcai.2018/438
20	PARK C， KIM D， HAN J， et al. Unsupervised attributed multiplex network embedding［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 5371-5378. 10.1609/aaai.v34i04.5985
21	WU Z H， PAN S R， CHEN F W， et al. A comprehensive survey on graph neural networks［J］. IEEE Transactions on Neural Networks and Learning Systems， 2021， 32（1）： 4-24. 10.1109/tnnls.2020.2978386
22	VELIČKOVIĆ P， CUCURULL G， CASANOVA A， et al. Graph attention network［EB/OL］. （2018-02-04）［2022-04-22］..
23	WANG X， WANG R J， SHI C， et al. Multi-component graph convolutional collaborative filtering［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 6267-6274. 10.1609/aaai.v34i04.6094
24	GAO H C， HUANG H. Deep attributed network embedding［C］// Proceedings of the 27th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2018： 3364-3370. 10.24963/ijcai.2018/467
25	PAN Y， HU G Y， QIU J Y， et al. FLGAI： a unified network embedding framework integrating multi-scale network structures and node attribute information［J］. Applied Intelligence， 2020， 50（11）： 3976-3989. 10.1007/s10489-020-01780-7

[1]	Zhiqiang ZHAO, Peihong MA, Xinhong HEI. Crowd counting method based on dual attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2886-2892.
[2]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[3]	Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738.
[4]	Kaipeng XUE, Tao XU, Chunjie LIAO. Multimodal sentiment analysis network with self-supervision and multi-layer cross attention [J]. Journal of Computer Applications, 2024, 44(8): 2387-2392.
[5]	Pengqi GAO, Heming HUANG, Yonghong FAN. Fusion of coordinate and multi-head attention mechanisms for interactive speech emotion recognition [J]. Journal of Computer Applications, 2024, 44(8): 2400-2406.
[6]	Zhonghua LI, Yunqi BAI, Xuejin WANG, Leilei HUANG, Chujun LIN, Shiyu LIAO. Low illumination face detection based on image enhancement [J]. Journal of Computer Applications, 2024, 44(8): 2588-2594.
[7]	Shangbin MO, Wenjun WANG, Ling DONG, Shengxiang GAO, Zhengtao YU. Single-channel speech enhancement based on multi-channel information aggregation and collaborative decoding [J]. Journal of Computer Applications, 2024, 44(8): 2611-2617.
[8]	Wu XIONG, Congjun CAO, Xuefang SONG, Yunlong SHAO, Xusheng WANG. Handwriting identification method based on multi-scale mixed domain attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2225-2232.
[9]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[10]	Dianhui MAO, Xuebo LI, Junling LIU, Denghui ZHANG, Wenjing YAN. Chinese entity and relation extraction model based on parallel heterogeneous graph and sequential attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2018-2025.
[11]	Li LIU, Haijin HOU, Anhong WANG, Tao ZHANG. Generative data hiding algorithm based on multi-scale attention [J]. Journal of Computer Applications, 2024, 44(7): 2102-2109.
[12]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[13]	Dahai LI, Zhonghua WANG, Zhendong WANG. Dual-branch low-light image enhancement network combining spatial and frequency domain information [J]. Journal of Computer Applications, 2024, 44(7): 2175-2182.
[14]	Wenliang WEI, Yangping WANG, Biao YUE, Anzheng WANG, Zhe ZHANG. Deep learning model for infrared and visible image fusion based on illumination weight allocation and attention [J]. Journal of Computer Applications, 2024, 44(7): 2183-2191.
[15]	Yan ZHOU, Yang LI. Rectified cross pseudo supervision method with attention mechanism for stroke lesion segmentation [J]. Journal of Computer Applications, 2024, 44(6): 1942-1948.

Attribute network representation learning with dual auto-encoder

双路自编码器的属性网络表示学习

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 25

Related Articles 15

Recommended Articles

Metrics