基于聚类分析分库策略的社交网络数据库查询性能与数据迁移

doi:10.11772/j.issn.1001-9081.2017.03.673

计算机应用 ›› 2017, Vol. 37 ›› Issue (3): 673-679.DOI: 10.11772/j.issn.1001-9081.2017.03.673

• 第四届大数据学术会议(CCF BIGDATA2016) • 上一篇下一篇

基于聚类分析分库策略的社交网络数据库查询性能与数据迁移

梁双, 周丽华, 杨培忠

云南大学信息学院, 昆明 650000

收稿日期:2016-09-26 修回日期:2016-10-23 出版日期:2017-03-10 发布日期:2017-03-22
通讯作者: 周丽华
作者简介:梁双(1987-),男,河南信阳人,硕士,主要研究方向:社交网络分析、分布式数据库;周丽华(1968-),女,云南昆明人,教授,博士,CCF会员,主要研究方向:数据挖掘、社交网络分析;杨培忠(1992-),男,云南保山人,硕士研究生,CCF会员,主要研究方向:数据挖掘、社交网络分析。
基金资助:
国家自然科学基金资助项目（61262069，61472346）；云南省自然科学基金资助项目（2016FA026，2015FB114，2015FB149）；云南大学中青年骨干教师、创新研究团队发展计划（XT412011）。

Query performance and data migration for social network database with shard strategy based on clustering analysis

LIANG Shuang, ZHOU Lihua, YANG Peizhong

College of Information, Yunnan University, Kunming Yunnan 650000, China

Received:2016-09-26 Revised:2016-10-23 Online:2017-03-10 Published:2017-03-22
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61262069, 61472346), the Natural Science Foundation of Yunnan Province (2015FB114, 2015FB149, 2016FA026), the Program for Young and Middle-aged Skeleton Teachers in Yunnan University, the Program for Innovation Research Team in Yunnan University (XT412011).

摘要/Abstract

摘要： 社交网络数据具有一定的聚合性，即特征上相近的用户之间更容易产生某种行为。依照常规的水平切分方法，在执行这些事件的信息查询时，将会耗费大量的时间和连接损耗去依次访问多个数据库。针对此问题，提出了基于聚类分析的社交网络数据库分库策略。将社交网络主体的特征标量进行聚类，使得聚集程度高的主体尽量分割到一个或尽可能少的几个分库中去，从而提高事件的查询效率，并在此基础上兼顾负载均衡与大数据迁移等问题。实验结果表明，该策略在社交网络的主流事件查询上都表现出不同程度的性能提升，最高提升程度达到23.4%，并且实现了局部最优负载均衡和零数据迁移。总的来说，基于聚类分析的社交网络数据库分库策略在提高查询效率、平衡负载以及大数据迁移可行性上，比传统水平切割分库有了相当的优势。

关键词: 社交网络, 数据库分库, 聚类分析, 查询性能, 数据迁移

Abstract: Social network data has a certain degree of aggregation, namely the similar users are more prone to the same behavior. According to the conventional horizontal database shard method, a large amount of time and connection loss were consumed in order to access a plurality of databases in turn when performing the information query of these events. In order to solve this problem, the database shard strategy based on clustering analysis was proposed. Through clustering the characteristic scalars of social network subjects, the main body with the high aggregation was divided into one or as possible libraries to improve the query efficiency of the events, and to give consideration to load balancing, large data migration and other issues. The experimental results show that for the mainstream social networking events, the performance improvement of the proposed strategy is up to 23.4% at most, and local optimal load balance and zero data migration are realized. In general, the database shard strategy based on clustering analysis of social network, has a considerable advantage on improving query efficiency, balance load balancing and large data migration feasibility over the traditional conventional horizontal database shard method of cutting library.

Key words: social network, database shard, clustering analysis, query performance, data migration

中图分类号:

TP302

梁双, 周丽华, 杨培忠. 基于聚类分析分库策略的社交网络数据库查询性能与数据迁移[J]. 计算机应用, 2017, 37(3): 673-679.

LIANG Shuang, ZHOU Lihua, YANG Peizhong. Query performance and data migration for social network database with shard strategy based on clustering analysis[J]. Journal of Computer Applications, 2017, 37(3): 673-679.

参考文献

[1] 肖凌,刘继红,姚建初.分布式数据库系统的研究与应用[J].计算机工程,2001,27(1):33-35.(XIAO L, LIU J H, YAO J C. Research and application of distributed database[J]. Computer Engineering, 2001, 27(1):33-35.)
[2] KARAGIANNIS T, GKANTSIDIS C, NARAYANAN D, et al. Hermes:clustering users in large-scale e-mail services[C]//Proceedings of the 1st ACM Symposium on Cloud Computing. New York:ACM, 2010:89-100.
[3] AGARWAL S, DUNAGAN J, JAIN N, et al. Volley:automated data placement for geo-distributed cloud services[C]//Proceedings of the 7th USENIX Conference on Networked Systems Design and Implementation. Berkeley, CA:USENIX Association, 2010:2.
[4] PUJOL J M, ERRAMILLI V, SIGANOS G, et al. The little engine(s) that could:scaling online social networks[C]//Proceedings of the ACM SIGCOMM 2010 Conference. New York:ACM, 2010:375-386.
[5] DUONG Q, GOEL S, HOFMAN J, et al. Sharding social networks[C]//WSDM'13:Proceedings of the 6th ACM International Conference on Web Search and Data Mining. New York:ACM, 2013:223-232.
[6] 熊华平,李莉娇,陈付平,等.大型异构数据库数据迁移系统的研究与应用[J].计算机应用与软件,2012,29(7):178-181.(XIONG H P, LI L J, CHEN F P, et al. Research and application of data migration system in large scale heterogeneous database[J]. Computer Applications and Software, 2012, 29(7):178-181.)
[7] 张友兰.大型信息系统可靠性试验最小样本量方法研究[J].环境技术,2013(11):59-64.(ZHANG Y L. Research on the reliability experiment minimum size sample method for large-scale information system[J]. Environmental Technology, 2013(11):59-64.)
[8] 李进芳,郭海明.样本量确定的经济学分析[J].统计与决策,2009(19):22-23.(LI J F, GUO H M. Economic analysis of sample size determination[J]. Statistics and Decision, 2009(19):22-23.)
[9] MISLOVE A, MARCON M, GUMMADI K P, et al. Measurement and analysis of online social networks[C]//Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement. New York:ACM, 2007:29-42.
[10] MOENS S, AKSEHIRLI E, GOETHALS B. Frequent itemset mining for big data[C]//Proceedings of the 2013 IEEE International Conference on Big Data. Piscataway, NJ:IEEE, 2013:111-118.
[11] FORTUNATO S. Community detection in graphs[J]. Physics Reports, 2010, 486(3/4/5):75-174.
[12] DENG J, DONG W, SOCHER R, et al. ImageNet:a large-scale hierarchical image database[C]//Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2009:248-255.
[13] HUANG J, ERTEKIN S, GILES C L. Efficient name disambiguation for large-scale databases[C]//Proceedings of the 10th European Conference on Principle and Practice of Knowledge Discovery in Databases. Berlin:Springer, 2006:536-544.

[1]	张萌, 李维华. 用户互动表示下的影响力最大化算法[J]. 计算机应用, 2021, 41(7): 1964-1969.
[2]	戴嫣然, 戴国庆, 袁玉波. 基于肤色学习的多人脸前景抽取方法[J]. 计算机应用, 2021, 41(6): 1659-1666.
[3]	郭佳, 韩李涛, 孙宪龙, 周丽娟. 自动确定聚类中心的比较密度峰值聚类算法[J]. 计算机应用, 2021, 41(3): 738-744.
[4]	孙鹤立, 徐统, 何亮, 贾晓琳. 融合用户历史行为与社交关系的个性化社交事件推荐方法[J]. 计算机应用, 2021, 41(2): 324-329.
[5]	赵旭剑, 王崇伟. 基于图卷积网络的微博新闻故事线抽取方法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3139-3144.
[6]	陈港, 孟相如, 康巧燕, 阳勇. 基于拓扑分割与聚类分析的虚拟软件定义网络映射算法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3309-3318.
[7]	李翔锟, 贾彩燕. 融合重叠社区正则化及隐式反馈的协同过滤方法[J]. 计算机应用, 2021, 41(1): 53-59.
[8]	盛俊, 李斌, 陈崚. 基于模块度和标签传递的推荐算法[J]. 计算机应用, 2020, 40(9): 2606-2612.
[9]	李春英, 汤庸, 肖政宏, 李天送. 学术社交网络中的权威学者推荐模型[J]. 计算机应用, 2020, 40(9): 2594-2599.
[10]	杨书新, 梁文, 朱凯丽. 社交网络中对立影响最大化算法[J]. 计算机应用, 2020, 40(7): 1944-1949.
[11]	易东义, 邓根强, 董超雄, 祝苗苗, 吕周平, 朱岁松. 基于图卷积神经网络的医保欺诈检测算法[J]. 计算机应用, 2020, 40(5): 1272-1277.
[12]	冯丽萍, 韩琦, 周志刚, 白增亮. 社交网络影响的不良信息扩散建模及最优控制策略[J]. 计算机应用, 2020, 40(3): 735-739.
[13]	单晓欢, 张志国, 宋宝燕, 任成林. 事件社交网中基于有向标签图及用户反馈的活动推荐方法[J]. 计算机应用, 2020, 40(2): 448-453.
[14]	何昊晨, 张丹红. 基于多维社交关系嵌入的深层图神经网络推荐方法[J]. 计算机应用, 2020, 40(10): 2795-2803.
[15]	孙鹤立, 何亮, 何方, 孙苗苗, 贾晓琳. 基于网络嵌入的稀疏子图发现算法[J]. 计算机应用, 2020, 40(10): 2929-2935.

基于聚类分析分库策略的社交网络数据库查询性能与数据迁移

Query performance and data migration for social network database with shard strategy based on clustering analysis

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics