Representation learning for topic-attention network

doi:10.11772/j.issn.1001-9081.2019081529

Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (2): 441-447.DOI: 10.11772/j.issn.1001-9081.2019081529

• CCF NDBC 2019 • Previous Articles Next Articles

Representation learning for topic-attention network

Jingfeng GUO¹^,², Hui DONG¹^,²(), Tingwei ZHANG¹^,², Xiao CHEN³

^1.College of Information Science and Engineering，Yanshan University，Qinhuangdao Hebei 066004，China
^2.Key Laboratory for Computer Virtual Technology and System Integration of Hebei Province，Qinhuangdao Hebei 066004，China
^3.Network Technology Center，Hebei Normal University of Science and Technology，Qinhuangdao Hebei 066004，China

Received:2019-08-12 Revised:2019-09-10 Accepted:2019-10-24 Online:2019-11-04 Published:2020-02-10
Contact: Hui DONG
About author:GUO Jingfeng， born in 1962， Ph. D.， professor. His research interests include database， data mining， social network analysis.
ZHANG Tingwei， born in 1995， M. S. candidate. His research interests include network representation learning， social network analysis.
CHEN Xiao， born in 1983， Ph. D.， research assistant. Her research interests include graph mining， network representation learning， social network analysis.
Supported by:
the National Natural Science Foundation of China(61472340);the National Youth Science Foundation of Hebei(F2017209070);the Doctoral Research Start-up Fund（Natural Science） of Hebei Normal University of Science and Technology(2019YB011);the Natural Science Foundation of Hebei(F2019203157);the Key Projects of Science and Technology Research in Colleges and Universities of Hebei Province(ZD2019004)

主题关注网络的表示学习

郭景峰¹^,², 董慧¹^,²(), 张庭玮¹^,², 陈晓³

^1.燕山大学信息科学与工程学院，河北秦皇岛 066004
^2.河北省计算机虚拟技术与系统集成重点实验室，河北秦皇岛 066004
^3.河北科技师范学院网络技术中心，河北秦皇岛 066004

通讯作者: 董慧
作者简介:郭景峰（1962—），男，黑龙江哈尔滨人，教授，博士，CCF会员，主要研究方向：数据库、数据挖掘、社会网络分析
张庭玮（1995—），男，河北秦皇岛人，硕士研究生，主要研究方向：网络表示学习、社会网络分析
陈晓（1983—），女，河北秦皇岛人，助理研究员，博士，CCF会员，主要研究方向：图挖掘、网络表示学习、社会网络分析。
基金资助:
国家自然科学基金资助项目(61472340);河北省青年科学基金资助项目(F2017209070);河北科技师范学院博士研究启动基金（自然科学）资助项目(2019YB011);河北省自然科学基金资助项目(F2019203157);河北省高等学校科学技术研究项目重点项目(ZD2019004)

Abstract

Abstract:

Concerning the problem that heterogeneous network representation learning only considers social relations in structure and ignores semantics， combining the social relationship between users and the preference of users for topics， a representation learning algorithm based on topic-attention network was proposed. Firstly， according to the characteristics of the topic-attention network and combining with the idea of the identical-discrepancy-contrary （determination and uncertainty） of set pair analysis theory， the transition probability model was given. Then， a random walk algorithm based on two types of nodes was proposed by using the transition probability model， so as to obtain the relatively high-quality random walk sequence. Finally， the embedding vector space representation of the topic-attention network was obtained by modeling based on two types of nodes in the sequences. Theoretical analysis and experimental results on the Douban dataset show that the random walk algorithm combined with the transition probability model is more comprehensive in analyzing the connection relationship between nodes in the network. The modularity of the proposed algorithm is 0.699 8 when the number of the communities is 13， which is nearly 5% higher than that of metapath2vec algorithm， and can capture more detailed information in the network.

Key words: topic-attention network, set pair analysis, transition probability, random walk, representation learning

摘要：

针对异质网络表示学习仅从结构方面考虑社交关系而忽略语义这一问题，结合用户间的社交关系和用户对主题的偏好两个方面，提出基于主题关注网络的表示学习算法。首先，针对主题关注网络的特点，结合集对分析理论的同异反（确定与不确定）思想，给出转移概率模型；然后，在转移概率模型的基础上提出了一种基于两类节点的随机游走算法，以得到相对高质量的随机游走序列；最后，基于序列中两类节点建模得到主题关注网络的嵌入向量空间表示。理论分析和在豆瓣数据集上的实验结果表明，结合转移概率模型的随机游走算法能更全面地分析网络中节点的连接关系，当划分社区的个数为13时，所提算法的模块度为0.699 8，相比metapath2vec算法提高了近5%，可以更详细地捕获网络中的信息。

关键词: 主题关注网络, 集对分析, 转移概率, 随机游走, 表示学习

CLC Number:

TP391

Jingfeng GUO, Hui DONG, Tingwei ZHANG, Xiao CHEN. Representation learning for topic-attention network[J]. Journal of Computer Applications, 2020, 40(2): 441-447.

郭景峰, 董慧, 张庭玮, 陈晓. 主题关注网络的表示学习[J]. 《计算机应用》唯一官方网站, 2020, 40(2): 441-447.

Figures/Tables 14

References 19

1	涂存超，杨成，刘知远，等. 网络表示学习综述［J］. 中国科学：信息科学， 2017， 47（8）： 980-996. 10.1360/n112017-00145
	TU C C， YANG C， LIU Z Y， et al. Network representation learning： an overview［J］. SCIENTIA SINICA Informationis， 2017， 47（8）： 980-996. 10.1360/n112017-00145
2	ZHAO Y， LIU Z， SUN M. Representation learning for measuring entity relatedness with rich information［C］// Proceedings of the 24th International Joint Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2015： 1412-1418. 10.1609/aaai.v33i01.3301817
3	TANG J， QU M， MEI Q. PTE： predictive text embedding through large-scale heterogeneous text networks［C］// Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2015： 1165-1174. 10.1145/2783258.2783307
4	CHEN H， YIN H， WANG W， et al. PME： projected metric embedding on heterogeneous networks for link prediction［C］// Proceeding of the 24th ACM SIGKDD International Conference on Knowledge Discover and Data Mining. New York： ACM， 2018： 1177-1186. 10.1145/3219819.3219986
5	DONG Y， CHAWLA N V， SWAMI A. metapath2vec： scalable representation learning for heterogeneous networks ［C］// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2017： 135-144. 10.1145/3097983.3098036
6	FU T Y， LEE W C， LEI Z. HIN2Vec： explore meta-paths in heterogeneous information networks for representation learning［C］// Proceeding of the 2017 ACM International Conference on Information and Knowledge Management. New York： ACM， 2017： 1797-1806. 10.1145/3132847.3132953
7	ZHU D， CUI P， WANG D， et al. Deep variational network embedding in Wasserstein space［C］// Proceeding of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2018： 2827-2836. 10.1145/3219819.3220052
8	DENG H， HAN J， ZHAO B， et al. Probabilistic topic models with biased propagation on heterogeneous information networks［C］// Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2011： 1271-1279. 10.1145/2020408.2020600
9	LI A Q， AHMED A， RAVI S， et al. Reducing the sampling complexity of topic models［C］// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2014： 891-900. 10.1145/2623330.2623756
10	陈晓，郭景峰，范超智. 基于联系度的主题关注网络社区发现方法研究［J］.计算机工程与应用，2017，53（17）：85-93. （CHEN X， GUO J F， FAN C Z. Community discovering based on connection degree in topic-attention network［J］. Journal of Computer Engineering and Applications， 2017： 53（17）： 85-93.）
11	SUN Y， HAN J， YAN X， et al. PathSim： meta path-based top-K similarity search in heterogeneous information networks［C］// Proceedings of the 37th International Conference on Very Large Data Bases. New York： ACM， 2011： 992-1003. 10.14778/3402707.3402736
12	PEROZZI B， AL-RFOU R， SKIENA S. DeepWalk： online learning of social representations［C］// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2014： 701-710. 10.1145/2623330.2623732
13	RONG X. word2vec parameter learning explained［EB/OL］. ［2018-05-05］. .
14	GROVER A， LESKOVEC J. Node2vec： scalable feature learning for networks［C］// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discover and Data Mining. New York： ACM， 2016： 855-864. 10.1145/2939672.2939754
15	TANG J， QU M， WANG M， et al. LINE： large-scale information network embedding［C］// Proceedings of the 24th International Conference on World Wide Web. New York： ACM， 2015： 1067-1077. 10.1145/2736277.2741093
16	许磊，黄玲，王昌栋.保持Motif结构的网络表示学习［J］.计算机科学与探索，2019，13（8）：1261-1271. 10.3389/fnbot.2019.00011
	XU L， HUANG L， WANG C D. Motif-preserving network representation learning［J］. Journal of Frontiers of Computer Science and Technology， 2019， 13（8）： 1261-1271. 10.3389/fnbot.2019.00011
17	WANG D， CUI P， ZHU W. Structural deep network embedding［C］// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2016： 1225-1234. 10.1145/2939672.2939753
18	SUN Y Z， NORICK B， HAN J W， et al. Integrating meta-path selection with user-guided object clustering in heterogeneous information networks［C］// Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2012： 1348-1356. 10.1145/2339530.2339738
19	赵克勤. 集对分析及其初步应用［M］. 杭州：浙江科学技术出版社， 2000： 1-198.
	ZHAO K Q. Set Pair Analysis and Preliminary Application of Set Pair［M］. Hangzhou： Zhejiang Science and Technology Press， 2000： 1-198.

节点对	（u₁，u₄）	（u₅，t₃）	（t₃，u₅）
S	1	1	1
∣CNT（u_i，u_j）₁∣	0	0	0
∣CNU（u_i，u_j）₁∣	2	2	0
∣CNT（u_i，u_j）_1∩2∣	1	0	0
∣CNT（u_i，u_j）_2∩1∣	1	0	0
∣CNT（u_i，u_j）₂∣	1	0	0

节点对	（u₁，u₄）	（u₅，t₃）	（t₃，u₅）
S	1	1	1
∣CNT（u_i，u_j）₁∣	0	0	0
∣CNU（u_i，u_j）₁∣	2	2	0
∣CNT（u_i，u_j）_1∩2∣	1	0	0
∣CNT（u_i，u_j）_2∩1∣	1	0	0
∣CNT（u_i，u_j）₂∣	1	0	0

v_current	u₂	u₃	u₄	u₅	u₆	t₁	t₂	t₃
u₁	0.19	0.08	0.18	0.05	0.06	0.12	0.23	0.10
t₃	0.00	0.25	0.25	0.25	0.25	0.00	0.00	0.00

v_current	u₂	u₃	u₄	u₅	u₆	t₁	t₂	t₃
u₁	0.19	0.08	0.18	0.05	0.06	0.12	0.23	0.10
t₃	0.00	0.25	0.25	0.25	0.25	0.00	0.00	0.00

v_next	P（v_next∣v_current）	v_next	P（v_next∣v_current）
u₅	0.029 847	u₉	0.025 764
u₆	0.029 360	u₁₀	0.005 381
u₇	0.029 360	u₁₁	0.029 847
u₈	0.034 477	u₁₂	0.020 588

Representation learning for topic-attention network

主题关注网络的表示学习

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 19

Related Articles 15

Recommended Articles

Metrics

算法	K
算法	5	6	7	8	9	10	11	12	13
DeepWalk	0.546 5	0.552 1	0.545 7	0.530 1	0.551 2	0.565 4	0.578 5	0.482 1	0.574 4
Node2vec	0.552 4	0.582 5	0.594 3	0.563 2	0.605 8	0.635 6	0.622 4	0.627 9	0.626 3
MPNE	0.499 5	0.554 8	0.572 1	0.572 3	0.558 8	0.584 6	0.629 5	0.617 7	0.634 5
PTE	0.551 7	0.586 2	0.593 3	0.587 6	0.602 5	0.639 6	0.628 7	0.639 5	0.645 8
matepath2vec	0.558 3	0.573 6	0.589 3	0.598 1	0.599 4	0.640 3	0.631 2	0.643 1	0.650 2
TANE	0.5665	0.5976	0.5970	0.612 7	0.6092	0.6435	0.6325	0.6568	0.6998

实验组序号m	α	β	γ₁	γ₂	δ	χ₁	χ₂	ε	θ	Q_m
1	0.75	0.25	0.00	0.00	0.00	0	1	0.7	0.3	0.63
2	0.40	0.30	0.10	0.10	0.10	1/3	2/3	0.7	0.3	0.70
3	0.25	0.25	1/6	1/6	1/6	1/2	1/2	0.7	0.3	0.52
4	0.10	0.30	0.20	0.20	0.20	2/3	1/3	0.7	0.3	0.37
5	0.00	0.25	0.25	0.25	0.25	1	0	0.7	0.3	0.16

数据集	α	β	γ₁	γ₂	δ	χ₁	χ₂	ε	θ
Karate	0.35	0.25	0.15	0.15	0.1	2/3	1/3	0.7	0.3
豆瓣	0.40	0.30	0.10	0.10	0.1	2/3	1/3	0.7	0.3

[1]	WANG Zisen, LIANG Ying, LIU Zhengjun, XIE Xiaojie, ZHANG Wei, SHI Hongzhou. Matching method for academic expertise of research project peer review experts [J]. Journal of Computer Applications, 2021, 41(8): 2418-2426.
[2]	TANG Andi, HAN Tong, XU Dengwu, XIE lei. Chaotic elite Harris hawks optimization algorithm [J]. Journal of Computer Applications, 2021, 41(8): 2265-2272.
[3]	ZHANG Meng, LI Weihua. Influence maximization algorithm based on user interactive representation [J]. Journal of Computer Applications, 2021, 41(7): 1964-1969.
[4]	LU Guoqing, SUN Hao. Two-dimensional mapping of swarm robot based on random walk [J]. Journal of Computer Applications, 2021, 41(7): 2121-2127.
[5]	YU Dunhui, ZHANG Luyi, ZHANG Xiaoxiao, MAO Liang. User recommendation method of cross-platform based on knowledge graph and restart random walk [J]. Journal of Computer Applications, 2021, 41(7): 1871-1877.
[6]	ZHANG Rong, ZHANG Xianguo. Opinion spam detection based on hierarchical heterogeneous graph attention network [J]. Journal of Computer Applications, 2021, 41(5): 1275-1281.
[7]	FAN Wei, WANG Huimin, XING Yan. Auto-encoder based multi-view attributed network representation learning model [J]. Journal of Computer Applications, 2021, 41(4): 1064-1070.
[8]	DU Xusheng, YU Jiong, YE Lele, CHEN Jiaying. Outlier detection algorithm based on graph random walk [J]. Journal of Computer Applications, 2020, 40(5): 1322-1328.
[9]	ZHENG Wenping, YUE Xiangdou, YANG Gui. Improved label propagation algorithm based on random walk [J]. Journal of Computer Applications, 2020, 40(12): 3423-3429.
[10]	ZHANG Zhao, JI Jianmin, CHEN Xiaoping. Adversarial negative sample generation for knowledge representation learning [J]. Journal of Computer Applications, 2019, 39(9): 2489-2493.
[11]	WANG Wentao, WU Lintao, HUANG Ye, ZHU Rongbo. Link prediction model based on densely connected convolutional network [J]. Journal of Computer Applications, 2019, 39(6): 1632-1638.
[12]	FAN Wei, XIE Cong, XIAO Chunjing, CAO Shuyan. Random walking recommendation algorithm based on combinational category space [J]. Journal of Computer Applications, 2019, 39(4): 984-988.
[13]	LIU Zhengming, MA Hong, LIU Shuxin, LI Haitao, CHANG Sheng. Network representation learning algorithm incorporated with node profile attribute information [J]. Journal of Computer Applications, 2019, 39(4): 1012-1020.
[14]	WANG Wentao, HUANG Ye, WU Lintao, KE Xuan, TANG Wan. Network representation learning algorithm based on improved random walk [J]. Journal of Computer Applications, 2019, 39(3): 651-655.
[15]	LIU Chengbin, ZHENG Wei, FAN Xin, YANG Fengyu. Hybrid defect prediction model based on network representation learning [J]. Journal of Computer Applications, 2019, 39(12): 3633-3638.

算法	向量维度d	算法	向量维度d
Deepwalk	64	PTE	100
node2vec	128	metapath2vec	100
MPNE	64	TANE	64

算法	向量维度d	算法	向量维度d
Deepwalk	64	PTE	100
node2vec	128	metapath2vec	100
MPNE	64	TANE	64