基于灰色关联分析的分布式协同过滤推荐算法

doi:10.11772/j.issn.1001-9081.2016.04.1054

计算机应用 ›› 2016, Vol. 36 ›› Issue (4): 1054-1059.DOI: 10.11772/j.issn.1001-9081.2016.04.1054

基于灰色关联分析的分布式协同过滤推荐算法

邱桂, 闫仁武

江苏科技大学计算机科学与工程学院, 江苏镇江 212003

收稿日期:2015-09-26 修回日期:2015-12-04 出版日期:2016-04-10 发布日期:2016-04-08
通讯作者: 邱桂
作者简介:邱桂(1990-),男,安徽宿州人,硕士研究生,主要研究方向:数据挖掘、大数据; 闫仁武(1962-),男,山东梁山人,副教授,硕士,主要研究方向:数据挖掘、信息融合。

Distributed collaborative filtering recommendation algorithm based on gray association analysis

QIU Gui, YAN Renwu

School of Computer Science and Engineering, Jiangsu University of Science and Technology, Zhenjiang Jiangsu 212003, China

Received:2015-09-26 Revised:2015-12-04 Online:2016-04-10 Published:2016-04-08

摘要/Abstract

摘要： 针对原始的基于用户(User-based)或基于评分项目(Item-based)的协同过滤推荐算法(CFR)大多采用"硬分类"式聚类,且具有数据稀疏性和可扩展性的问题,提出一种基于灰色关联分析的分布式协同过滤推荐算法。算法使用Hadoop分布式计算平台,首先,计算评分矩阵中每个评分项目的灰色关系系数;然后,计算各评分项目的灰色关联度(GRG);最后,根据GRG获得每个评分项目的近邻集合,对不同用户的待预测项目用对应的近邻集合对其评分进行预测。通过在MovieLens数据集上进行实验,与User-based和Item-based的CFR算法相比,该算法平均绝对误差分别下降了1.07%和0.06%,而且随着数据规模的扩展,通过增加集群节点,算法运行效率有相应的提升。实验结果表明,该推荐算法可以有效地实现大规模数据的推荐,并能解决数据可扩展性的问题。

关键词: 灰色系统, 灰色关联度, 协同过滤推荐算法, 分布式系统, Hadoop

Abstract: In order to solve the problems of "hard classification" clustering, data sparsity and scalability in user-based or item-based Collaborative Filtering Recommendation (CFR) algorithms, a distributed collaborative filtering recommendation algorithm based on gray association analysis was proposed. Based on Hadoop platform, the grey relational coefficient of each item in rating matrix was calculated at first, then the Grey Relational Grade (GRG) of each item was calculated. Finally, the similar items for each item was constructed according to GRG, and item's rating for different users with related similar items was predicted. The experiment was conducted on the MovieLens dataset. The results showed that the Mean Absolute Error (MAE) of proposed algorithm was reduced by 1.07% and 0.06% respectively compared to the user-based and item-based CFR algorithms; and with the scale of dataset expending, the running efficiency was also improved by adding datanode to the Hadoop cluster. The experimental results illustrate that the proposed algorithm can make effective recommendation for large scale dataset and solve the problem of data scalability.

Key words: grey system, Grey Relational Grade (GRG), Collaborative Filtering Recommendation (CFR) algorithm, distributed system, Hadoop

中图分类号:

TP301.6

邱桂, 闫仁武. 基于灰色关联分析的分布式协同过滤推荐算法[J]. 计算机应用, 2016, 36(4): 1054-1059.

QIU Gui, YAN Renwu. Distributed collaborative filtering recommendation algorithm based on gray association analysis[J]. Journal of Computer Applications, 2016, 36(4): 1054-1059.

参考文献

[1] GOLDBERG D, NICHOLS D, OKI B M, et al. Using collaborative filtering to weave an information tapestry[J]. Communications of the ACM, 1992, 35(12):61-70.
[2] 杨志文, 刘波.基于Hadoop平台协同过滤推荐算法[J].计算机系统应用, 2013, 22(7):108-112.(YANG Z W, LIU B. Hadoop-based collaborative filtering recommendation algorithm[J]. Computer System & Applications, 2013, 22(7):108-112.)
[3] SARWAR B M. Sparsity, scalability, and distribution in recommender systems[D]. Twin Cities:University of Minnesota, 2001:168.
[4] PATIL V A, RAGHA L. Comparing performance of collaborative filtering algorithms[C]//Proceedings of the 2012 International Conference on Communication, Information & Computing Technology. Piscataway, NJ:IEEE, 2012:1-6.
[5] ZHAO Z, SHANG M. User-based collaborative-filtering recommendation algorithms on Hadoop[C]//WKDD 2010:Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining. Piscataway, NJ:IEEE, 2010:478-481.
[6] 赵宏霞, 王新海, 杨皎平.基于用户和项目因子分析的混合协同推荐算法[J]. 计算机应用, 2011, 31(5):1382-1390.(ZHAO H X, WANG X H, YANG J P. Mixed collaborative recommendation algorithm based on factor analysis of user and item[J]. Journal of Computer Applications, 2011, 31(5):1382-1390.)
[7] 范波, 程久军.用户间多相似度协同过滤推荐算法[J].计算机科学, 2012, 39(1):23-26.(FAN B, CHENG J J. Collaborative filtering recommendation algorithm based on user's multi-similarity[J]. Computer Science, 2012, 39(1):23-26.)
[8] PAUL R, MEOPHYTOS I, MITESH S, et al. GroupLens:An open architecture for collaborative filtering of netnews[C]//Proceedings of ACM 1994 Conference on Computer Supported Cooperative Work. New York:ACM, 1994:175-186.
[9] BADRUL S, GEORGE K, JOSEPH K, et al. Item-based collaborative filtering recommendation algorithms[C]//Proceedings of the 10th International Conference on World Wide Web. New York:ACM, 2001:285-295.
[10] 邓聚龙.灰理论基础[M].武汉:华中科技大学出版社, 2002:2,122.(DENG J L. Basis of Grey Theory[M]. Wuhan:Huazhong University of Science & Technology Press, 2002:2,122.)
[11] The Apache Software Foundation. The Apache Hadoop project[CP/OL].[2015-09-12]. http://hadoop.apache.org/.
[12] TOM W. Hadoop权威指南[M].2版.周敏奇, 王晓玲, 译.北京:清华大学出版社, 2011:15,411.(TOM W. Hadoop:the Definitive Guide[M]. 2nd ed. ZHOU M Q, WANG X L, translated. Beijing:Tsinghua University Press, 2011:15,411.)
[13] DEAN J, GHEMAWAT S. MapReduce:simplified data processing on large clusters[C]//OSDI 2004:Proceedings of the 6th Conference on Symposium on Opearting Systems Design & Implementation. Berkeley, CA:USENIX Association, 2004, 6:10.
[14] The Apache Software Foundation. The Apache HBase project[CP/OL].[2015-09-12]. http://hbase.apache.org/.
[15] METROPOLIS N, ULAM S. The Monte Carlo method[J]. Journal of the American Statistical Association, 1953, 60(247):335-341.
[16] ZHANG J, LIN Z, XIAO B, et al. An optimized item-based collaborative filtering recommendation algorithm[C]//IC-NIDC 2009:Proceedings of the 2009 IEEE International Conference on Network Infrastructure and Digital Content. Piscataway, NJ:IEEE, 2009:414-418.
[17] 孙楠军, 刘天时.基于项目分类和用户群体兴趣的协同过滤算法[J]. 计算机工程与应用, 2015, 51(10):128-136.(SUN N J, LIU T S. Collaborative filtering based on item classification and user group interest[J]. Computer Engineering and Applications, 2015, 51(10):128-136.)
[18] 刘思峰, 党耀国, 方志耕, 等.灰色系统理论及其应用[M].北京:科学出版社, 2004.(LIU S F, DANG Y G, FANG Z G, et al. Theory and Application of Grey System[M]. Beijing:Science Press, 2004.)

基于灰色关联分析的分布式协同过滤推荐算法

Distributed collaborative filtering recommendation algorithm based on gray association analysis

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	吴志军, 王航. 基于订阅/发布服务的广域信息管理系统应急响应机制[J]. 计算机应用, 2020, 40(5): 1340-1347.
[2]	董聪, 张晓, 程文迪, 石佳. 基于新型存储器件的分布式文件系统性能优化[J]. 计算机应用, 2020, 40(12): 3594-3603.
[3]	李耘书, 滕飞, 李天瑞. 基于微操作的Hadoop参数自动调优方法[J]. 计算机应用, 2019, 39(6): 1589-1594.
[4]	郑振涛, 赵卓峰, 王桂玲, 徐垚. 面向港口停留区域识别的船舶停留轨迹提取方法[J]. 计算机应用, 2019, 39(1): 113-117.
[5]	廖天星, 王玲. 融合项目标签相似性的协同过滤推荐算法[J]. 计算机应用, 2018, 38(4): 1007-1011.
[6]	周岳, 陈庆奎. 面向大规模数据接入系统的负载平衡机制[J]. 计算机应用, 2018, 38(1): 50-55.
[7]	李强, 刘晓峰. 基于Hopfield神经网络的云存储负载均衡策略[J]. 计算机应用, 2017, 37(8): 2214-2217.
[8]	吴家皋, 夏轩, 刘林峰. 基于MapReduce的轨迹压缩并行化方法[J]. 计算机应用, 2017, 37(5): 1282-1286.
[9]	温占考, 易秀双, 田申申, 李婕, 王兴伟. 基于边界矩阵低阶近似和近邻模型的协同过滤算法[J]. 计算机应用, 2017, 37(12): 3472-3476.
[10]	肖圣龙, 陈昕, 李卓. 面向社会安全事件的分布式神经网络攻击行为分类方法[J]. 计算机应用, 2017, 37(10): 2794-2798.
[11]	付晨, 钟诚, 叶波. MapReduce并行加速数据流多模式相似性搜索[J]. 计算机应用, 2017, 37(1): 37-41.
[12]	杨俊杰, 廖卓凡, 冯超超. 大数据存储架构和算法研究综述[J]. 计算机应用, 2016, 36(9): 2465-2471.
[13]	邢淑凝, 刘方爱, 赵晓晖. 基于聚类划分的高效用模式并行挖掘算法[J]. 计算机应用, 2016, 36(8): 2202-2206.
[14]	朱洁, 李雯睿, 王江平, 赵红. 基于节点集计算能力差异的Hadoop自适应任务调度算法[J]. 计算机应用, 2016, 36(4): 918-922.
[15]	刘青, 付印金, 倪桂强, 梅建民. 基于Hadoop平台的分布式重删存储系统[J]. 计算机应用, 2016, 36(2): 330-335.