Graph representation learning by autoencoder with one-shot aggregation

doi:10.11772/j.issn.1001-9081.2021101860

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (1): 8-14.DOI: 10.11772/j.issn.1001-9081.2021101860

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Graph representation learning by autoencoder with one-shot aggregation

YUAN Lining¹, LIU Zhao²

1.School of Information and Cyber Security， People’s Public Security University of China， Beijing 100038， China
2.Graduate School， People’s Public Security University of China， Beijing 100038， China

Received:2021-11-02 Revised:2021-12-15 Online:2021-12-31
Contact: LIU Zhao， born in 1981， Ph. D.， lecturer. His research interests include machine learning， computer vision.
About author:YUAN Lining， born in 1995， M. S. candidate. His research interests include machine learning， graph neural network；
Supported by:
This work is partially supported by Fundamental Research Funds for the Central Universities （2019JKF425）， National Key Research and Development Program of China （2020YFC1522600）.

基于One-Shot聚合自编码器的图表示学习

袁立宁¹, 刘钊²

1.中国人民公安大学信息网络安全学院，北京 100038
2.中国人民公安大学研究生院，北京 100038

通讯作者: 刘钊（1981—），男，北京人，讲师，博士，主要研究方向：机器学习、计算机视觉。liuzhao@ppsuc.edu.cn
作者简介:袁立宁（1995—），男，河北唐山人，硕士研究生，CCF会员，主要研究方向：机器学习、图神经网络；；
基金资助:
中央高校基本科研业务费专项（2019JKF425）；国家重点研发计划项目（2020YFC1522600）。

Abstract

Abstract: AutoEncoder （AE） is an efficient learning model for graph data representation， but most of the Graph AutoEncoders （GAEs） are shallow models， their efficiencies decrease with the increase of hidden layers. Aiming at the above problems， a new GAE model OSA-GAE and a new Variational Graph AutoEncoder OSA-VGAE were proposed based on One-Shot Aggregation （OSA） and Exponential Linear Unit （ELU）. Firstly， the encoder was constructed by a multi-layer Graph Convolutional Network （GCN）， and OSA and ELU function were introduced. Then， the topology of the graph was recovered by the inner product decoder in the decoding stage. In addition， a regularization term was introduced to the loss function in order to prevent parameter overfitting during the model training process. Experimental results show that OSA and ELU function can improve the performance and gradient information transmission of the deep GAEs. In the link prediction task of benchmark citation dataset PubMed， when using 6-layer GCN， deep OSA-VGAE improves Area Under ROC Curve （AUC） and Average Precision （AP） by 8.67 and 6.85 percentage points respectively compared to the original VGAE， and deep OSA-GAE improves AP and AUC by 6.82 and 4.39 percentage points respectively compared to the original GAE.

Key words: AutoEncoder (AE), Graph AutoEncoder (GAE), Graph Convolutional Network (GCN), One-Shot Aggregation (OSA), link prediction

摘要： 自编码器（AE）是一种高效的图数据表示学习模型，但大多数图自编码器（GAE）为浅层模型，其效率会随着隐藏层的增加而降低。针对上述问题，提出基于One-Shot聚合（OSA）和指数线性（ELU）函数的GAE模型OSA-GAE和图变分自编码器模型OSA-VGAE。首先，利用多层图卷积网络（GCN）构建编码器，并引入OSA和ELU函数；然后，在解码阶段使用内积解码器恢复图的拓扑结构；此外，为了防止模型训练过程中的参数过拟合，在损失函数中引入正则化项。实验结果表明，OSA和ELU函数可以有效提高深层GAE的性能，改善模型的梯度信息传递。在使用6层GCN时，基准引文数据集PubMed的链接预测任务中，深层OSA-VGAE相较于原始的VGAE在ROC曲线下的面积（AUC）和平均精度（AP）上分别提升了8.67和6.85个百分点，深层OSA-GAE相较于原始的GAE在AP和AUC上分别提升了6.82和4.39个百分点。

关键词: 自编码器, 图自编码器, 图卷积网络, One-Shot聚合, 链接预测

CLC Number:

TP183

YUAN Lining, LIU Zhao. Graph representation learning by autoencoder with one-shot aggregation[J]. Journal of Computer Applications, 2023, 43(1): 8-14.

袁立宁, 刘钊. 基于One-Shot聚合自编码器的图表示学习[J]. 《计算机应用》唯一官方网站, 2023, 43(1): 8-14.

References

1 BHAGAT S， CORMODE G， MUTHUKRISHNAN S. Node classification in social networks［M］// AGGARWAL C. Social Network Data Analytics. Boston： Springer， 2011：115-148. 10.1007/978-1-4419-8462-3_5
2 刘昱阳，李龙杰，单娜，等. 融合聚集系数的链接预测方法［J］. 计算机应用， 2020， 40（1）： 28-35. 10.11772/j.issn.1001-9081.2019061008 LIU Y Y， LI L J， SHAN N， et al. Link prediction method fusing clustering coefficients［J］. Journal of Computer Applications， 2020， 40（1）： 28-35. 10.11772/j.issn.1001-9081.2019061008
3 尤坊州，白亮. 关键节点选择的快速图聚类算法［J］. 计算机科学与探索， 2021， 15（10）： 1930-1937. 10.3778/j.issn.1673-9418.2007004 YOU F Z， BAI L. Fast graph clustering algorithm based on selection of key nodes［J］. Journal of Frontiers of Computer Science and Technology， 2021， 15（10）： 1930-1937. 10.3778/j.issn.1673-9418.2007004
4 LeCUN Y， BENGIO Y， HINTON G. Deep learning［J］. Nature， 2015， 521（7553）： 436-444. 10.1038/nature14539
5 GOYAL P， FERRARA E. Graph embedding techniques， applications， and performance： a survey［J］. Knowledge-Based Systems， 2018， 151： 78-94. 10.1016/j.knosys.2018.03.022
6 刘思，刘海，陈启买，等. 基于网络表示学习与随机游走的链路预测算法［J］. 计算机应用， 2017， 37（8）： 2234-2239. 10.11772/j.issn.1001-9081.2017.08.2234 LIU S， LIU H， CHEN Q M， et al. Link prediction algorithm based on network representation learning and random walk［J］. Journal of Computer Applications， 2017， 37（8）： 2234-2239. 10.11772/j.issn.1001-9081.2017.08.2234
7 BOURLARD H， KAMP Y. Auto-association by multilayer perceptrons and singular value decomposition［J］. Biological Cybernetics， 1988， 59（4/5）： 291-294. 10.1007/bf00332918
8 PEROZZI B， AL-RFOU R， SKIENA S. DeepWalk： online learning of social representations ［C］// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2014： 701-710. 10.1145/2623330.2623732
9 GROVER A， LESKOVEC J. node2vec： Scalable feature learning for networks［C］// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2016： 855-864. 10.1145/2939672.2939754
10 HUNG P T， YAMANISHI K. Word2vec skip-gram dimensionality selection via sequential normalized maximum likelihood［J］. Entropy， 2021， 23（8）： No.997. 10.3390/e23080997
11 WU Z H， PAN S R， CHEN F W， et al. A comprehensive survey on graph neural networks［J］. IEEE Transactions on Neural Networks and Learning Systems， 2021， 32（1）： 4-24. 10.1109/tnnls.2020.2978386
12 LI G H， MüLLER M， QIAN G C， et al. DeepGCNs： making GCNs go as deep as CNNs［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021（Early Access）： 1-1. 10.1109/tpami.2021.3074057
13 LEE Y， HWANG J W， LEE S， et al. An energy and GPU-computation efficient backbone network for real-time object detection［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2019： 752-760. 10.1109/cvprw.2019.00103
14 CLEVERT D A， UNTERTHINER T， HOCHREITER S. Fast and accurate deep network learning by Exponential Linear Units （ELUs）［EB/OL］. （2016-02-22）［2021-12-10］.https：//arxiv.org/pdf/1511.07289.pdf.
15 PEROZZI B， KULKARNI V， SKIENA S. Walklets： multiscale graph embeddings for interpretable network classification［EB/OL］. （2016-05-06）［2021-12-10］.https：//arxiv.org/pdf/1605.02115v1.pdf. 10.1145/3110025.3110086
16 WANG D X， CUI P， ZHU W W. Structural deep network embedding［C］// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2016： 1225-1234. 10.1145/2939672.2939753
17 CAO S S， LU W， XU Q K. Deep neural networks for learning graph representations［C］// Proceedings of the 30th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2016： 1145-1152. 10.1609/aaai.v30i1.10179
18 KIPF T N， WELLING M. Variational graph auto-encoders［EB/OL］. （2016-11-21）［2021-09-15］.https：//arxiv.org/pdf/1611.07308.pdf.
19 KINGMA D P， WELLING M. Auto-encoding variational Bayes［EB/OL］. （2014-05-01）［2021-11-19］.https：//arxiv.org/pdf/1312.6114.pdf. 10.1561/2200000056
20 NALLBANI I， AYANZADEH A， KESER R K， et al. Representation learning using graph autoencoders with residual connections［EB/OL］. （2021-05-03）［2021-10-22］.https：//arxiv.org/pdf/2105.00695.pdf.
21 HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
22 XIAO Y， XIAO D， HU B B， et al. ANE： network embedding via adversarial autoencoders［C］// Proceedings of the 2018 IEEE International Conference on Big Data and Smart Computing. Piscataway： IEEE， 2018： 66-73. 10.1109/bigcomp.2018.00019
23 MAKHZANI A， SHLENS J， JAITLY N， et al. Adversarial autoencoders［EB/OL］. （2016-05-25）［2021-12-10］.https：//arxiv.org/pdf/1511.05644.pdf.
24 HUANG G， LIU Z， MAATEN L van der， et al. Densely connected convolutional networks［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2261-2269. 10.1109/cvpr.2017.243
25 GU J X， WANG Z H， KUEN J， et al. Recent advances in convolutional neural networks［J］. Pattern Recognition， 2018， 77： 354-377. 10.1016/j.patcog.2017.10.013
26 GLOROT X， BORDES A， BENGIO Y. Deep sparse rectifier neural networks［C］// Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. New York： JMLR.org， 2011： 315-323.
27 KIPF T N， WELLING M. Semi-supervised classification with graph convolutional networks［EB/OL］. （2017-02-22）［2021-11-08］.https：//arxiv.org/pdf/1609.02907.pdf. 10.48550/arXiv.1609.02907
28 HAMMOND D K， VANDERGHEYNST P， GRIBONVAL R. Wavelets on graphs via spectral graph theory［J］. Applied and Computational Harmonic Analysis， 2011， 30（2）：129-150. 10.1016/j.acha.2010.04.005
29 KULLBACK S， LEIBLER R A. On information and sufficiency［J］. The Annals of Mathematical Statistics， 1951， 22（1）：79-86. 10.1214/aoms/1177729694
30 SEN P， NAMATA G， BILGIC M， et al. Collective classification in network data［J］. AI Magazine， 2008， 29（3）： 93-106. 10.1609/aimag.v29i3.2157
31 SALHA G， HENNEQUIN R， VAZIRGIANNIS M. Keep it simple： graph autoencoders without graph convolutional networks［EB/OL］. （2019-10-02）［2021-12-10］.https：//arxiv.org/pdf/1910.00942.pdf. 10.1016/j.neunet.2021.04.015
32 VELI?KOVI? P， CUCURULL G， CASANOVA A， et al. Graph attention networks［EB/OL］. （2018-02-04）［2021-10-10］.https：//arxiv.org/pdf/1710.10903.pdf.

[1]	Chuanlin PANG, Rui TANG, Ruizhi ZHANG, Chuan LIU, Jia LIU, Shibo YUE. Distributed power allocation algorithm based on graph convolutional network for D2D communication systems [J]. Journal of Computer Applications, 2024, 44(9): 2855-2862.
[2]	Guixiang XUE, Hui WANG, Weifeng ZHOU, Yu LIU, Yan LI. Port traffic flow prediction based on knowledge graph and spatio-temporal diffusion graph convolutional network [J]. Journal of Computer Applications, 2024, 44(9): 2952-2957.
[3]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[4]	Shibin LI, Jun GONG, Shengjun TANG. Semi-supervised heterophilic graph representation learning model based on Graph Transformer [J]. Journal of Computer Applications, 2024, 44(6): 1816-1823.
[5]	Longtao GAO, Nana LI. Aspect sentiment triplet extraction based on aspect-aware attention enhancement [J]. Journal of Computer Applications, 2024, 44(4): 1049-1057.
[6]	Xianfeng YANG, Yilei TANG, Ziqiang LI. Aspect-level sentiment analysis model based on alternating‑attention mechanism and graph convolutional network [J]. Journal of Computer Applications, 2024, 44(4): 1058-1064.
[7]	Kaitian WANG, Qing YE, Chunlei CHENG. Classification method for traditional Chinese medicine electronic medical records based on heterogeneous graph representation [J]. Journal of Computer Applications, 2024, 44(2): 411-417.
[8]	Zucheng WU, Xiaojun WU, Tianyang XU. Image-text retrieval model based on intra-modal fine-grained feature relationship extraction [J]. Journal of Computer Applications, 2024, 44(12): 3776-3783.
[9]	Xinrong HU, Jingxue CHEN, Zijian HUANG, Bangchao WANG, Xun YAO, Junping LIU, Qiang ZHU, Jie YANG. Graph convolution network-based masked data augmentation [J]. Journal of Computer Applications, 2024, 44(11): 3335-3344.
[10]	Nengqiang XIANG, Xiaofei ZHU, Zhaoze GAO. Information diffusion prediction model of prototype-aware dual-channel graph convolutional neural network [J]. Journal of Computer Applications, 2024, 44(10): 3260-3266.
[11]	Yanbo LI, Qing HE, Shunyi LU. Aspect sentiment triplet extraction integrating semantic and syntactic information [J]. Journal of Computer Applications, 2024, 44(10): 3275-3280.
[12]	Wanting JI, Wenyi LU, Yuhang MA, Linlin DING, Baoyan SONG, Haolin ZHANG. Machine reading comprehension event detection based on relation-enhanced graph convolutional network [J]. Journal of Computer Applications, 2024, 44(10): 3288-3293.
[13]	Hui JIANG, Qiuyan YAN, Zhujun JIANG. Symmetric positive definite autoencoder method for multivariate time series anomaly detection [J]. Journal of Computer Applications, 2024, 44(10): 3294-3299.
[14]	Hanxiao SHI, Leichun WANG. Short-term power load forecasting by graph convolutional network combining LSTM and self-attention mechanism [J]. Journal of Computer Applications, 2024, 44(1): 311-317.
[15]	Yi ZHANG, Gangsheng CAI, Zhenmei WANG. Long non-coding RNA-disease association prediction model based on semantic and global dual attention mechanism [J]. Journal of Computer Applications, 2023, 43(7): 2125-2132.

Graph representation learning by autoencoder with one-shot aggregation

基于One-Shot聚合自编码器的图表示学习

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics