%0 Journal Article %A GUO Binglei %A LIAO Bin %A LIU Yan %A YU Jiong %A ZHANG Tao %A ZHANG Xuguang %T Performance optimization of ItemBased recommendation algorithm based on Spark %D 2017 %R 10.11772/j.issn.1001-9081.2017.07.1900 %J Journal of Computer Applications %P 1900-1905 %V 37 %N 7 %X Under MapReduce computing scenarios, complex data mining algorithms typically require multiple MapReduce jobs' collaboration process to compete the task. However, serious redundant disk read and write and repeat resource request operations among multiple MapReduce jobs seriously degrade the performance of the algorithm under MapReduce. To improve the computational efficiency of ItemBased recommendation algorithm, firstly, the performance issues of the ItemBased collaborative filtering algorithm under MapReduce platform were analyzed. Secondly, the execution efficiency of the algorithm was improved by taking advantage of Spark's performance superiority on iterative computation and memory computing, and the ItemBased collaborative filtering algorithm under Spark platform was implemented. The experimental results show that, when the size of the cluster nodes is 10 and 20, the running time of the algorithm in Spark is only 25.6% and 30.8% of that in MapReduce. The algorithm's overall computing efficiency of Spark platform improves more than 3 times compared with that of MapReduce platform. %U http://www.joca.cn/EN/10.11772/j.issn.1001-9081.2017.07.1900