关键词: 微信公众号, Hadoop平台, MapReduce模型, KNN, 分类

Abstract: Abstract: People who engage in WeChat commercials extract valuable information inefficiently when WeChat subscription data grows rapidly. To resolve the issue, a method of classifying and labeling the WeChat subscription data parallel is proposed. Firstly, with the premise that introduces practical application of WeChat subscription, the shortcoming of single node based on KNN classification algorithm is analysed. Then, the distributed KNN algorithm on the Hadoop platform using MapReduce application model is implemented, the efficiency between stand-alone and distributed algorithm is contrasted and K value is tune. In experiment, the training sample set is specified, text similarity between testing sample and training sample is divided into word segmentation, feature words extraction, weight calculation, cosine coefficient calculation. On the basis of the 24 categories, after the experiment of classifying ten million truthful data, every WeChat subscription is set single label or multi label, the classification accuracy after optimization reaches 82 percentage points, the number of the WeChat subscription associated with the life accounts for more than 70 percentage points. The research shows the transformation rate of information has been improved when used the classification results, the distributed KNN algorithm has higher efficiency and robustness than the single node algorithm for WeChat subscription data.

Key words: WeChat subscription, Hadoop platform, MapReduce model, KNN, classification